search

bcgov / FIT_opendatadownloader

Monitor open data, report when changes are detected
Apache License 2.0
0 stars 0 forks source link

page through ESRI map service results #8

Closed smnorris closed 2 months ago

smnorris commented 2 months ago

It seems the easiest way to get data available via an esri map service is via the api, requesting geojson. A portal will offer download options but getting a link (and getting ogr to follow the redirects to s3/az blob) seems tricky - using the api is simple. But responses will generally have a maxRecordCount, requests need to be paged.

Guides: https://medium.com/@jesse.b.nestler/how-to-extract-every-feature-from-an-esri-map-service-using-python-b6e34743574a https://community.esri.com/t5/python-questions/extract-data-from-open-data/m-p/1265161/highlight/true#M67006

Example dataset: https://data-cityofpg.opendata.arcgis.com/datasets/CityofPG::road-centerline API request: https://services2.arcgis.com/CnkB6jCzAsyli34z/arcgis/rest/services/OpenData_RoadTraffic/FeatureServer/4/query?outFields=*&where=1%3D1&f=geojson

smnorris commented 2 months ago

also: https://github.com/openaddresses/esri-dump https://github.com/openaddresses/pyesridump

smnorris commented 2 months ago

pyesridump works well for the PG example, no need to add a custom implementation.