scrape CKAN for data sources

[josh-chamberlain]

Context

CKAN is one of the most popular open-source data portals, and is often used by governments to publish police data.

We have the start of a CKAN scraper here: https://github.com/Police-Data-Accessibility-Project/scrapers/tree/main/scrapers_library/data_portals/ckan/package_list

Requirements

• [ ] scrape a batch of potential portals or sources from CKAN.
  • target one of our record types
    • calls for service or dispatch logs are almost universally useful and frequently available
  • or check our list of data sources record type
    • related issue: https://github.com/Police-Data-Accessibility-Project/data-source-identification/issues/63
• [ ] optional: run them through one of our ML models to see how it classifies them: https://huggingface.co/PDAP
• [ ] Submit them to our Data Sources database (data dictionary here)
  • they can be uploaded as a CSV to airtable for now
• Based on this, we can choose what to do next, how to scale, how many to attempt

[CaptainStabs]

CKAN API Docs: https://docs.ckan.org/en/2.9/api/