At the moment we just pass a general query and sync the data to GH so we can check a diff when something changes. If it changes, we know to update the scrapers. It would be useful to add a base class like we have for CKAN and Datapress that can parse that data a bit more intelligently and do it for us.
There are a number of councils where we are calling
https://opendata.arcgis.com/api/v2/datasets
to identify new data e.g:At the moment we just pass a general query and sync the data to GH so we can check a diff when something changes. If it changes, we know to update the scrapers. It would be useful to add a base class like we have for CKAN and Datapress that can parse that data a bit more intelligently and do it for us.
Tunbridge Wells is a starting point for this:
https://github.com/wdiv-scrapers/DC-PollingStations-TunbridgeWells/blob/master/scraper.py