search

openaddresses / pyesridump

Scrapes an ESRI MapServer REST endpoint to spit out more generally-usable geodata.
MIT License
322 stars 68 forks source link

Can't download this arcgis map - retry pause #81

Open latot opened 2 years ago

latot commented 2 years ago

Hi, I was trying to get a map, but always end in retry pause 0, 0, 1, 2, 3....

Layer: https://services3.arcgis.com/Dhl01RVOOnbjTdY7/ArcGIS/rest/services/COBERTURAS_ADJUDICADAS_5G/FeatureServer/11

The layer is working, we can access and use it from: https://www.arcgis.com/home/webmap/viewer.html?url=https://services3.arcgis.com/Dhl01RVOOnbjTdY7/ArcGIS/rest/services/COBERTURAS_ADJUDICADAS_5G/FeatureServer&source=sd

JSON is working, we can get data with the next arguments:

https://services3.arcgis.com/Dhl01RVOOnbjTdY7/ArcGIS/rest/services/COBERTURAS_ADJUDICADAS_5G/FeatureServer/11/query?f=json&returnGeometry=true&spatialRel=esriSpatialRelIntersects&geometry={"xmin":-8140237.764258992,"ymin":-4461476.466945019,"xmax":-8061966.24729499,"ymax":-4383204.949981019,"spatialReference":{"wkid":102100,"latestWkid":3857}}&geometryType=esriGeometryEnvelope&inSR=102100&outFields=*&returnCentroid=false&returnExceededLimitFeatures=false&maxRecordCountFactor=3&outSR=102100&resultType=tile&quantizationParameters={"mode":"view","originPosition":"upperLeft","tolerance":152.87405657031263,"extent":{"xmin":-8140237.764258992,"ymin":-4461476.466945019,"xmax":-8061966.247294992,"ymax":-4383204.949981019,"spatialReference":{"wkid":102100,"latestWkid":3857}}}

No idea why ends like this, I try checking debug info, higher timeout, the --paginate-oid, and always ends the same.

Thx.

Geremia commented 10 months ago

I get this issue, too:

$ esri2geojson -v https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0 maricopa.json
2023-12-31 15:32:43,697 - cli.esridump - DEBUG - GET https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0, args {'f': 'json'}
2023-12-31 15:32:43,861 - cli.esridump - DEBUG - GET https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'where': '1=1', 'returnCountOnly': 'true', 'f': 'json'}
2023-12-31 15:32:55,231 - cli.esridump - INFO - Built 1726 requests using resultOffset method
2023-12-31 15:32:55,232 - cli.esridump - DEBUG - POST https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'resultOffset': 0, 'resultRecordCount': 1000, 'where': '1=1', 'geometryPrecision': 7, 'returnGeometry': True, 'outSR': '4326', 'outFields': '*', 'f': 'json'}
2023-12-31 15:33:35,391 - cli.esridump - INFO - retry pause 0
2023-12-31 15:33:35,391 - cli.esridump - DEBUG - POST https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'resultOffset': 0, 'resultRecordCount': 1000, 'where': '1=1', 'geometryPrecision': 7, 'returnGeometry': True, 'outSR': '4326', 'outFields': '*', 'f': 'json'}
2023-12-31 15:34:25,556 - cli.esridump - INFO - retry pause 1
2023-12-31 15:34:25,556 - cli.esridump - DEBUG - POST https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'resultOffset': 0, 'resultRecordCount': 1000, 'where': '1=1', 'geometryPrecision': 7, 'returnGeometry': True, 'outSR': '4326', 'outFields': '*', 'f': 'json'}
2023-12-31 15:35:25,741 - cli.esridump - INFO - retry pause 2
2023-12-31 15:35:25,741 - cli.esridump - DEBUG - POST https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'resultOffset': 0, 'resultRecordCount': 1000, 'where': '1=1', 'geometryPrecision': 7, 'returnGeometry': True, 'outSR': '4326', 'outFields': '*', 'f': 'json'}
2023-12-31 15:36:35,937 - cli.esridump - INFO - retry pause 3
2023-12-31 15:36:35,938 - cli.esridump - DEBUG - POST https://gis.mcassessor.maricopa.gov/arcgis/rest/services/Parcels/MapServer/0/query, args {'resultOffset': 0, 'resultRecordCount': 1000, 'where': '1=1', 'geometryPrecision': 7, 'returnGeometry': True, 'outSR': '4326', 'outFields': '*', 'f': 'json'}
2023-12-31 15:37:56,154 - cli.esridump - INFO - retry pause 4
Traceback (most recent call last):
  File "/home/alan/Downloads/ersidump/bin/esri2geojson", line 8, in <module>
    sys.exit(main())
  File "/home/alan/Downloads/ersidump/lib64/python3.9/site-packages/esridump/cli.py", line 119, in main
    feature = next(feature_iter)
  File "/home/alan/Downloads/ersidump/lib64/python3.9/site-packages/esridump/dumper.py", line 460, in __iter__
    raise download_exception
esridump.errors.EsriDownloadError: ('Could not connect to URL', ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='gis.mcassessor.maricopa.gov', port=443): Read timed out. (read timeout=30)")))
iandees commented 10 months ago

Timeouts like this are likely from something that's rate limiting you, or the server is otherwise unable to keep up with your request rate.