wdiv-scrapers / dc-base-scrapers

https://morph.io/wdiv-scrapers/
MIT License
0 stars 0 forks source link

Add base class for ArcGIS Open Data v2 API #3

Open chris48s opened 5 years ago

chris48s commented 5 years ago

There are a number of councils where we are calling https://opendata.arcgis.com/api/v2/datasets to identify new data e.g:

At the moment we just pass a general query and sync the data to GH so we can check a diff when something changes. If it changes, we know to update the scrapers. It would be useful to add a base class like we have for CKAN and Datapress that can parse that data a bit more intelligently and do it for us.

Tunbridge Wells is a starting point for this:

https://github.com/wdiv-scrapers/DC-PollingStations-TunbridgeWells/blob/master/scraper.py

GeoWill commented 3 years ago

This should probably be v3 now. Cheatsheet: https://gist.github.com/jgravois/1b7ec5080e992a59f65cf7a2190e4365