search

Esri / arcgis-python-api

Documentation and samples for ArcGIS API for Python
https://developers.arcgis.com/python/
Apache License 2.0
1.89k stars 1.1k forks source link

gis.content.add(), rate limit of around 25/30 items every two hours? (HTTP Error 403: Forbidden) #1177

Closed maxrostron closed 2 years ago

maxrostron commented 2 years ago

Describe the bug I've written a python script which loops through around 400 folders on Google Drive, gets the various GeoJson and Shp files, creates a shareable link, and add's them as individual items on ArcGis Online.

This script works perfectly as intended. However, after around 25-30 items are added to AGOL, I start getting a forbidden error and the next items loops are not added.

I then can't run the script again for another hour or two. There are thousands of files, so this isn't ideal.

Does anyone know what's causing this? Is there a rate limit on adding items, and is there a way around this?

To Reproduce Steps to reproduce the behavior: Loop through a folder containing a hundred or so Shapefiles and GeoJsons, using a script similar to below.

gis = GIS("https://www.arcgis.com", "USERNAME", "PASSWORD")
try:
  new_item = gis.content.add(data=download_link, item_properties = file_properties, folder = drive_folder['title'])
except Exception as e:
  print(e)

error:

HTTP Error 403: Forbidden

Expected behavior I was hoping to be able to loop through all 400 drive folders, adding the items to their equivalent folders in AGOL.

achapkowski commented 2 years ago

We'll explore the issue and have reached out to the ArcGIS Online team to see if they changed something. Thank you for reporting this.

achapkowski commented 2 years ago

@maxrostron I create a simulation of add/removing over 90 items in a 10 minute span. I could not reproduce the issue.

I used a mix of shp, fgdb, geojson, and csv files. I did not publish them to HFL because you said it was just adding.

I strongly recommend you reach out to support and provide them your workflow and date/times you experienced this issue. They can get more information specific to your case. I think this is a REST issue, not a Python API issue.

Thank you