Esri / overwrite-hosted-features

The Transportation 511 Script is used to keep the contents of a Feature Collection up to date. Please see
http://links.esri.com/stategovernment/help/transportation511
Apache License 2.0
7 stars 5 forks source link

Orphaned Files #18

Closed NikkiGolding closed 6 years ago

NikkiGolding commented 6 years ago

Overwrite/Upload of current information (.gdb) to Online fails due to orphaned files within ArcGIS Online. The files that are orphaned are low level AGO files and are not visible within the UI. KY DOT has run into this issue twice now.

Note: The cause of the files being orphaned has occurred when an update was occurring and Online encountered an issue. The repro steps are not simple to recreate the root cause.

When an attempt is made to update the gdb a message is returned:

Traceback info: File "overwrite_hosted_features.py", line 548, in run self._upload_fgdb() File "overwrite_hosted_features.py", line 427, in _upload_fgdb response = self._url_request(url, request_parameters, files=files, error_text='Failed to upload file geodatabase') File "overwrite_hosted_features.py", line 298, in _url_request raise Exception("{0}: {1}".format(error_text, response_json))

Error Info: Failed to upload file geodatabase: {u'error': {u'message': u"Item 'WazeAlerts.gdb.zip' already exists.", u'code': 409, u'details': []}} Elapsed Time: 0:00:15.265000

additional context below:

We believe the issue was introduced when the Online infrastructure was under significant load. Due to the significant load, there were resulting issues across the system. I presume the script was in the middle of a delete operation when the 'blip' occurred. This left the KY files in a state where they were inaccessible to the script in a way we were searching for them.

Moving forward, we can make changes to the script's delete operation to "bypass the indexes" to try to ensure this does not happen again. However, we will need to test the scripts more thoroughly before we can release them. There is a fairly large caveat, we wont have another repro case to test the new version of the script unless the above circumstances with the Online infrastructure repeat.

Note: There is a fairly easy workaround for the issue if we ever get in a pinch again, we can rename the files being uploaded and alter the names for the files in the script.

jessnoo commented 6 years ago

verified on prd with a Solution test case that created the error.