scrape MuckRock for data sources

[josh-chamberlain]

Context

We have a lot of potential data sources in MuckRock, and we have a simple scraper for them.

Example of potentially highly relevant URLs: https://www.muckrock.com/foi/list/?page=1&per_page=100&q=use+of+force

Also, we request a lot of things. It would cool to one day import successful requests to our database: https://www.muckrock.com/foi/organization-list/?sort=date_submitted&order=desc

Requirements

• [ ] Scrape batch of requests (20? 100?) from MuckRock. Target one of our record types, or target Allegheny County PA.
• [ ] optional: run them through one of our ML models to see how it classifies them: https://huggingface.co/PDAP
• [ ] Submit them to our Data Sources database (data dictionary here)
  • they can be uploaded as a CSV to airtable for now
• Based on this, we can choose what to do next, how to scale, how many to attempt

[CaptainStabs]

Muckrock API docs: https://www.muckrock.com/api/

[josh-chamberlain]

@bonjarlow just assigning you since you mentioned you were looking at it! I also adjusted the issue to make space for an allegheny county target rather than record type.