blob file upload failed 5 times a week

[baosoul]

Describe the bug

An error occurred when uploading blob files

Related command

az storage blob upload --file log-2024.07.22.04.00.gz --container-name logs/2024/07/22/04 --account-name logs --auth-mode login

Errors

• az storage blob upload --file log-2024.07.22.04.00.gz --container-name logs/2024/07/22/04 --account-name logs --auth-mode login WARNING: Command group 'az storage' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Alive[ ] 0.4911%ERROR: The command failed with an unexpected error. Here is the traceback: ERROR: [Errno 11] Resource temporarily unavailable Traceback (most recent call last): File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 664, in execute raise ex File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 731, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 723, in _run_job return cmd_copy.exception_handler(ex) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler first(ex) File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler first(ex) File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/_exception_handler.py", line 17, in file_related_exception_handler raise ex File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 701, in _run_job result = cmd_copy(params) ^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 334, in call return self.handler(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler return op(command_args) ^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 642, in upload_blob response = client.upload_blob(data=stream, length=length, metadata=metadata, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_blob_client.py", line 746, in upload_blob return upload_block_blob(options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_upload_helpers.py", line 167, in upload_block_blob block_ids = upload_substream_blocks( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 113, in upload_substream_blocks range_ids = _parallel_uploads(executor, uploader.process_substream_block, upload_tasks, running_futures) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 30, in _parallel_uploads range_ids.extend([chunk.result() for chunk in done]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 30, in range_ids.extend([chunk.result() for chunk in done]) ^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/concurrent/futures/_base.py", line 449, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/concurrent/futures/_base.py", line 401, in get_result raise self._exception File "/opt/az/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 233, in process_substream_block return self._upload_substream_block_with_progress(block_data[0], block_data[1]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 239, in _upload_substream_block_with_progress range_id = self._upload_substream_block(index, block_stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 272, in _upload_substream_block self.service.stage_block( File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_generated/operations/_block_blob_operations.py", line 1222, in stage_block pipeline_response = self._client._pipeline.run( # type: ignore # pylint: disable=protected-access ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 213, in run return first_node.send(pipeline_request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_redirect.py", line 181, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/policies.py", line 520, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication.py", line 113, in send self.on_request(request) File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication.py", line 90, in on_request self._token = self._credential.get_token(self._scopes) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/credential_adaptor.py", line 65, in gettoken token, = self._get_token(scopes, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/credential_adaptor.py", line 38, in _get_token token = self._credential.get_token(*scopes, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/msal_authentication.py", line 69, in get_token result = self.acquire_token_silent_with_error(list(scopes), self._account, claims_challenge=claims, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1284, in acquire_token_silent_with_error return _clean_up(self._acquire_token_silent_with_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1307, in _acquire_token_silent_with_error result = self._acquire_token_silent_from_cache_and_possibly_refresh_it( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1438, in _acquire_token_silent_from_cache_and_possibly_refresh_it result = self._acquire_token_silent_by_finding_rt_belongs_to_me_or_my_family( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1499, in _acquire_token_silent_by_finding_rt_belongs_to_me_or_my_family last_resp = at = self._acquire_token_silent_by_finding_specific_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1547, in _acquire_token_silent_by_finding_specific_refresh_token response = client.obtain_token_by_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 830, in obtain_token_by_refresh_token resp = super(Client, self).obtain_token_by_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 262, in obtain_token_by_refresh_token return self._obtain_token("refresh_token", data=data, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oidc.py", line 170, in _obtain_token ret = super(Client, self)._obtain_token(grant_type, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 788, in _obtain_token (on_obtaining_tokens or self.on_obtaining_tokens)({ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1552, in on_obtaining_tokens=lambda event: self.token_cache.add(dict( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 374, in add super(SerializableTokenCache, self).add(event, kwargs) File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 180, in add return self.add(event, now=now) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 244, in __add self.modify(self.CredentialType.ACCESS_TOKEN, at, at) File "/opt/az/lib/python3.11/site-packages/msal_extensions/token_cache.py", line 63, in modify with CrossPlatLock(self._lock_location): File "/opt/az/lib/python3.11/site-packages/msal_extensions/cache_lock.py", line 58, in enter file_handle = self._lock.enter() ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/portalocker/utils.py", line 153, in enter__ return self.acquire() ^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/portalocker/utils.py", line 254, in acquire raise exceptions.LockException(exception) portalocker.exceptions.LockException: [Errno 11] Resource temporarily unavailable To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

Issue script & Debug output

• az storage blob upload --file log-2024.07.22.04.00.gz --container-name logs/2024/07/22/04 --account-name logs --auth-mode login WARNING: Command group 'az storage' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Alive[ ] 0.4911%ERROR: The command failed with an unexpected error. Here is the traceback: ERROR: [Errno 11] Resource temporarily unavailable Traceback (most recent call last): File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 664, in execute raise ex File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 731, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 723, in _run_job return cmd_copy.exception_handler(ex) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler first(ex) File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler first(ex) File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/_exception_handler.py", line 17, in file_related_exception_handler raise ex File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 701, in _run_job result = cmd_copy(params) ^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 334, in call return self.handler(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler return op(command_args) ^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/storage/operations/blob.py", line 642, in upload_blob response = client.upload_blob(data=stream, length=length, metadata=metadata, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_blob_client.py", line 746, in upload_blob return upload_block_blob(options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_upload_helpers.py", line 167, in upload_block_blob block_ids = upload_substream_blocks( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 113, in upload_substream_blocks range_ids = _parallel_uploads(executor, uploader.process_substream_block, upload_tasks, running_futures) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 30, in _parallel_uploads range_ids.extend([chunk.result() for chunk in done]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 30, in range_ids.extend([chunk.result() for chunk in done]) ^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/concurrent/futures/_base.py", line 449, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/concurrent/futures/_base.py", line 401, in get_result raise self._exception File "/opt/az/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 233, in process_substream_block return self._upload_substream_block_with_progress(block_data[0], block_data[1]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 239, in _upload_substream_block_with_progress range_id = self._upload_substream_block(index, block_stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/uploads.py", line 272, in _upload_substream_block self.service.stage_block( File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_generated/operations/_block_blob_operations.py", line 1222, in stage_block pipeline_response = self._client._pipeline.run( # type: ignore # pylint: disable=protected-access ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 213, in run return first_node.send(pipeline_request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_redirect.py", line 181, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/multiapi/storagev2/blob/v2022_11_02/_shared/policies.py", line 520, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/_base.py", line 70, in send response = self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication.py", line 113, in send self.on_request(request) File "/opt/az/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication.py", line 90, in on_request self._token = self._credential.get_token(self._scopes) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/credential_adaptor.py", line 65, in gettoken token, = self._get_token(scopes, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/credential_adaptor.py", line 38, in _get_token token = self._credential.get_token(*scopes, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/auth/msal_authentication.py", line 69, in get_token result = self.acquire_token_silent_with_error(list(scopes), self._account, claims_challenge=claims, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1284, in acquire_token_silent_with_error return _clean_up(self._acquire_token_silent_with_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1307, in _acquire_token_silent_with_error result = self._acquire_token_silent_from_cache_and_possibly_refresh_it( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1438, in _acquire_token_silent_from_cache_and_possibly_refresh_it result = self._acquire_token_silent_by_finding_rt_belongs_to_me_or_my_family( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1499, in _acquire_token_silent_by_finding_rt_belongs_to_me_or_my_family last_resp = at = self._acquire_token_silent_by_finding_specific_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1547, in _acquire_token_silent_by_finding_specific_refresh_token response = client.obtain_token_by_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 830, in obtain_token_by_refresh_token resp = super(Client, self).obtain_token_by_refresh_token( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 262, in obtain_token_by_refresh_token return self._obtain_token("refresh_token", data=data, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oidc.py", line 170, in _obtain_token ret = super(Client, self)._obtain_token(grant_type, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/oauth2cli/oauth2.py", line 788, in _obtain_token (on_obtaining_tokens or self.on_obtaining_tokens)({ File "/opt/az/lib/python3.11/site-packages/msal/application.py", line 1552, in on_obtaining_tokens=lambda event: self.token_cache.add(dict( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 374, in add super(SerializableTokenCache, self).add(event, kwargs) File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 180, in add return self.add(event, now=now) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/msal/token_cache.py", line 244, in __add self.modify(self.CredentialType.ACCESS_TOKEN, at, at) File "/opt/az/lib/python3.11/site-packages/msal_extensions/token_cache.py", line 63, in modify with CrossPlatLock(self._lock_location): File "/opt/az/lib/python3.11/site-packages/msal_extensions/cache_lock.py", line 58, in enter file_handle = self._lock.enter() ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/portalocker/utils.py", line 153, in enter__ return self.acquire() ^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/portalocker/utils.py", line 254, in acquire raise exceptions.LockException(exception) portalocker.exceptions.LockException: [Errno 11] Resource temporarily unavailable To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

Expected behavior

An error occurred when uploading blob files

Environment Summary

azure-cli 2.62.0

core 2.62.0 telemetry 1.1.0

Extensions: storage-preview 1.0.0b2

Dependencies: msal 1.28.1 azure-mgmt-resource 23.1.1

Python location '/opt/az/bin/python3' Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.11.8 (main, Jul 4 2024, 02:27:42) [GCC 9.4.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[calvinhzy]

Seems like a token issue, can you try to rerunning az login. Another thing to try is to run without the extension by running az extension remove -n storage-preview

[baosoul]

Seems like a token issue, can you try to rerunning az login. Another thing to try is to run without the extension by running az extension remove -n storage-preview

After the upload fails, run the command again to upload successfully. crontab is uploaded every hour az storage blob upload --file log-2024.07.22.04.00.gz --container-name logs/2024/07/22/04 --account-name logs --auth-mode login

[calvinhzy]

I am aware that the token needs to be refreshed every hour so maybe multiple processes try to compete for the lock? How are you running the crontab? And if you run jobs back to back within an hour, does it fail with this error?