search

r-three / common-pile

Repo to hold code and track issues for the collection of permissively licensed data
MIT License
22 stars 6 forks source link

Regulations.gov #32

Open nkandpa2 opened 11 months ago

nkandpa2 commented 11 months ago

Federal regulation proposals and user comments.

StellaAthena commented 10 months ago

@nkandpa2 is there any support you need?

nkandpa2 commented 10 months ago

I have the bandwidth to make some progress on this this week. An update of where I'm currently at:

Regulations.gov rate limits their API too aggressively to collect data in a reasonable amount of time. However, they do offer "curated downloads" where you request data from any one agency over a date range up to one year. To test out the code, I've requested and received data for ~20 agencies from the year 2022. This should be enough data to get started on development. Once the code is in a good spot and committed, I'll submit a bunch of requests to get data from these agencies for the past 20-30 years.