Blue sky law changes

This page summarizes the main dataset on blue sky law changes extracted from historical Blue Sky Law Reports (1948-1980). This data is used to construct the Blue Sky Law Index.

Download the blue sky law changes data (Github)

Data source

The primary data source is the Blue Sky Law Reports. These reports were released on a bi-weekly basis starting in 1946, published originally by Commerce Clearing House, Inc. Here is the summary from the first volume in 1946:

The two new Compilation Volumes of the Blue Sky Law Reporter which you have received are designed to afford complete and accurate answers to the two important questions confronting the blue sky practitioner:
(1) How are private securities issues qualified under state law; and (2) In what states may banks and trust companies, fiduciaries, and insurance companies invest in these securities issues?

Today’s growing securities market has increased the need for authoritative information on state blue sky regulation. The new Reporter is sent to you in answer to this need and in the hope that it will prove an invaluable aid in the solution of your securities problems arising under state laws and regulations.

Blue Sky Law Reporter, Report 1, July 1, 1946

Here is a screenshot of the original files:

Data construction: scanning, processing and cleaning

To create the data on changes, we took the following steps.

Scanning to pdf

Each available Blue Sky Reports document was manually scanned to pdf. These scans are not available because of copyright. Screenshots of several are available throughout the site. Each pdf has a report number and date.

PDF to textual data

Using standard OCR methods, we next extracted the pdfs to textual data. OCR technology has limitations, so each textual output was manually corrected to best match the pdf. There remain some errors in the data, but we believe that over 85% of errors have been corrected. At this stage, we can define our unit of observation. This is an event summarizing a legislative change, regulatory update, court decision, or federal event. An example of such an observation is here:

Data structure and variables

The resulting data has the following structure in figure and text form:

  • index
    • The report number
  • year, month and day
    • The year, month and day of the report. Reports were typically released every 2 weeks
  • title1
    • the section header the categorizes the changes
  • state
    • if available, the state related to the change
  • title2
    • the change-specific summary of the change
  • text
    • the paragraph or more of the text summarizing the change

Data summary statistics

  • number of rows: 7153
  • number of rows with non-missing state: 6665
  • number of rows with non-missing title1: 6214
  • number of rows with non-missing title2: 6214
  • year coverage: there are some gaps in the data where Blue Sky Law Reports were unavailable
Data coverage of the blue sky law changes for the full sample
Figure 1: Mentions of various changes over time
Figure 2: Mentions of “exempt(ion)” over time for states with most changes
Figure 3: Mentions of “registration”” over time for states with most changes
Figure 4: Changes related to “exemption” across states and over time.
Figure 5: Most common words associated with “registration” changes
Figure 6: Most common words associated with “broker” or “dealer” changes

Limitations

The data has some limitations that we hope to be addressed in the next year:

  • annual coverage in gaps: several years of data were unavailable. The first pass to fill these gaps is to collect the state laws for each state. Ultimately, we need to translate differences in these laws into 1-2 paragraph summaries that mimic the Blue Sky Law Reports.
  • scanning and textual translation: there are still some minor textual issues with the OCR to text.
  • Better categorization of changes: some rows of the data are not specifically state-level blue sky law changes.