search

Azure / azure-storage-python

Microsoft Azure Storage Library for Python
https://azure-storage.readthedocs.io
MIT License
338 stars 240 forks source link

status=409 error (container is being deleted) is not retired by built-in retry classes #569

Closed rfernand2 closed 5 years ago

rfernand2 commented 5 years ago

Which service(blob, file, queue) does this issue concern?

blob

Which version of the SDK was used? Please provide the output of pip freeze.

azure-batch==6.0.0 azure-common==1.1.18 azure-storage-blob==1.4.0 azure-storage-common==1.4.0 azure-storage-file==1.4.0

What problem was encountered?

After deleting a container, I tried to create another container with the same name. If the time interval is between these actions is short (under 30 secs or so), the error "container is being deleted" is issued. I tried setting client.retry to LinearRetry().retry, but the error is still not retried.

Analysis

lines 72-81 in azure-storage-common/azure/storage/common/retry.py show that status=409 is handled by returning False (never retry). This is a common problem (for my code at least) - I recommend that it be enabled for retries.

Stack Trace

tClient-Request-ID=08146a00-53be-11e9-8644-309c23b63418 Retry policy did not allow for a retry: Server-Timestamp=Sun, 31 Mar 2019 14:05:23 GMT, Server-Request-ID=5553e8f2-e01e-00de-60ca-e798e6000000, HTTP status code=409, Exception=The specified container is being deleted. Try operation later. ErrorCode: ContainerBeingDeleted<?xml version="1.0" encoding="utf-8"?>ContainerBeingDeletedThe specified container is being deleted. Try operation later.RequestId:5553e8f2-e01e-00de-60ca-e798e6000000Time:2019-03-31T14:05:23.4952262Z. Traceback (most recent call last): File "xt_test.py", line 114, in store_test(xt) File "xt_test.py", line 21, in store_test xt.create_workspace(ws_name) File "c:\github\ExperimentTools\xt\xt_store.py", line 65, in create_workspace self.helper.create_workspace(ws_name, description) File "c:\github\ExperimentTools\xt\xt_azure_store_blob.py", line 88, in create_workspace self.error("could not create workspace: " + ws_name) File "c:\github\ExperimentTools\xt\xt_azure_store_blob.py", line 52, in error raise Exception(f"Error - {msg}") Exception: Error - could not create workspace: ws1

Have you found a mitigation/solution?

Use a custom retry callback to retry this error.

Note: for table service, please post the issue here instead: https://github.com/Azure/azure-cosmosdb-python.

zezha-msft commented 5 years ago

Hi @rfernand2, thanks for the feedback!

Error 409 might be returned in other scenarios which might persist longer, so I don't think that this change should be made to the general retry policy. You are welcome to create your own retry policy, and set it on the client object, like so.