Azure / azure-storage-python

Microsoft Azure Storage Library for Python
https://azure-storage.readthedocs.io
MIT License
338 stars 240 forks source link

TypeError thrown during request signing #519

Closed mawah closed 5 years ago

mawah commented 5 years ago

Which service(blob, file, queue) does this issue concern?

Blob.

Which version of the SDK was used? Please provide the output of pip freeze.

azure-cli-command-modules-nspkg==2.0.2
azure-cli-nspkg==3.0.3
azure-common==1.1.14
azure-mgmt-eventgrid==0.4.0
azure-mgmt-nspkg==2.0.0
azure-nspkg==2.0.0
azure-storage==0.36.0

What problem was encountered?

Creation of a BlockBlobService using my storage account name and credentials returned without error:

bbs = BlockBlobService(account_name=storage_account_name, account_key=storage_account_key)

However, when I attempt to create a blob, I get the following error:

with open('filename', 'rb') as f:
    bbs.create_blob_from_bytes(container_name=container_name, blob_name=blob_name, blob=f.read())
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
C:\Anaconda3\lib\site-packages\azure\storage\storageclient.py in _perform_request(self, request, parser, parser_args, operation_context)
    215                     _add_date_header(request)
--> 216                     self.authentication.sign_request(request)
    217 

C:\Anaconda3\lib\site-packages\azure\storage\_auth.py in sign_request(self, request)
     68             self._get_canonicalized_headers(request) + \
---> 69             self._get_canonicalized_resource(request) + \
     70             self._get_canonicalized_resource_query(request)

C:\Anaconda3\lib\site-packages\azure\storage\_auth.py in _get_canonicalized_resource(self, request)
     35         uri_path = request.path.split('?')[0]
---> 36         return '/' + self.account_name + uri_path
     37 

TypeError: must be str, not tuple

During handling of the above exception, another exception occurred:

AzureException                            Traceback (most recent call last)
C:\Anaconda3\lib\site-packages\azure\storage\storageclient.py in _perform_request(self, request, parser, parser_args, operation_context)
    251                         # Automatic chaining in Python 3 means we keep the trace
--> 252                         raise AzureException(ex.args[0])
    253                     else:

AzureException: must be str, not tuple

The connection seems to be retried indefinitely, and I was forced to stop execution manually, leading to the following exception:

During handling of the above exception, another exception occurred:

KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-26-621e146df55f> in <module>()
      1 with open('sample_articles/0001/lvl2k20181019055148.jpg', 'rb') as f:
----> 2     bbs.create_blob_from_bytes(container_name=container_name, blob_name=blob_name, blob=f.read())

C:\Anaconda3\lib\site-packages\azure\storage\blob\blockblobservice.py in create_blob_from_bytes(self, container_name, blob_name, blob, index, count, content_settings, metadata, validate_content, progress_callback, max_connections, lease_id, if_modified_since, if_unmodified_since, if_match, if_none_match, timeout)
    694             if_none_match=if_none_match,
    695             timeout=timeout,
--> 696             use_byte_buffer=True
    697         )
    698 

C:\Anaconda3\lib\site-packages\azure\storage\blob\blockblobservice.py in create_blob_from_stream(self, container_name, blob_name, stream, count, content_settings, metadata, validate_content, progress_callback, max_connections, lease_id, if_modified_since, if_unmodified_since, if_match, if_none_match, timeout, use_byte_buffer)
    525                 if_match=if_match,
    526                 if_none_match=if_none_match,
--> 527                 timeout=timeout)
    528 
    529             if progress_callback:

C:\Anaconda3\lib\site-packages\azure\storage\blob\blockblobservice.py in _put_blob(self, container_name, blob_name, blob, content_settings, metadata, validate_content, lease_id, if_modified_since, if_unmodified_since, if_match, if_none_match, timeout)
    914             request.headers['Content-MD5'] = _to_str(computed_md5)
    915 
--> 916         return self._perform_request(request, _parse_base_properties)
    917 
    918     def _put_block(self, container_name, blob_name, block, block_id,

C:\Anaconda3\lib\site-packages\azure\storage\storageclient.py in _perform_request(self, request, parser, parser_args, operation_context)
    276 
    277                     # Sleep for the desired retry interval
--> 278                     sleep(retry_interval)
    279                 else:
    280                     raise ex

KeyboardInterrupt: 

Have you found a mitigation/solution?

Not yet. My Microsoft alias is mawah if you'd like to reach out for debugging.

Note: for table service, please post the issue here instead: https://github.com/Azure/azure-cosmosdb-python.

zezha-msft commented 5 years ago

Hi @mawah, thanks for reaching out!

From the stack trace, it seems to me that your account_name might be the problem, could you please verify whether it is indeed a str?

I also noticed that you are using a very old version of the SDK. If you don't mind, could you please upgrade to the latest version(azure-storage-blob)? Note that the library was split into separate services, please refer to this section of the README.

mawah commented 5 years ago

I checked what the init method for the _ServiceParameters class in connection.py was receiving: by the time it arrives there, the account name is a single-element tuple, ('myaccountname', ). I find that odd given that I've provided a simple string to the BlockBlobService constructor as shown above; haven't figured out where the issue is introduced. I'm using Python 3.6.5, if it makes a difference.

As a workaround, I manually edited the _ServiceParameters class's init method to store the tuple's first value in the account_name attribute. This seems to have gotten me past that problem. I'm now checking into an apparently separate issue: my storage account is in the government cloud, and I think the connection strings need to be constructed a little bit differently than in commercial.

mawah commented 5 years ago

Also just tried switching to azure-storage-blob, and this fixed the problem with the tuple vs. string error as well. Sorry, didn't realize that the pip package name had changed!