search

Police-Data-Accessibility-Project / scrapers

Code relating to scraping public police data.
https://pdap.io
GNU General Public License v3.0
157 stars 33 forks source link

initialize directory and readme #175

Closed rainmana closed 2 years ago

rainmana commented 2 years ago

Not sure if I'm doing this correctly, Josh! But I want to add my code here for "PDAP TuCan" which is what I have so far. It does the following:

  1. Takes in input at the command line for a base CKAN URL
  2. Appends the API URI that gets the packages list from the API for a CKAN instance (v3)
  3. Provides error checking for that URL and issues an error to standard out and closes if you goof up your URL entry
  4. Gets the list of packages from the CKAN instances and adds them to a dictionary data structure
  5. Counts the number of packages available at the CKAN URL
  6. Optional (commented out at the moment) prints the dictionary of packages available

Next steps:

  1. Presents user with an opportunity to display the packages/maybe save them as a CSV

  2. Allows users to choose which packages they want to download

  3. Downloads them and their metadata in the format presented

  4. Continue to add color coding, display consistency, and make this accessible to non-coders/people like me that need color coding for extra neurodivergent readability (I may look at the Wooey Python project or Gooey to make it even more so)

  5. Add documentation from already existing comments and make sure code is "over-commented" as well as add a README

  6. Add a module that posts data to the CKAN API for our instance

  7. Make this module portable but include the option to download data from one CKAN URL as designed above, then copy to our instance

  8. Success?

Let me know if I'm missing anything!

josh-chamberlain commented 2 years ago

closing in favor of #176