Documentation for block_ids results in double-base64 encoding

[stewartadam]

Which service(blob, file, queue) does this issue concern?

Blob

Which version of the SDK was used? Please provide the output of pip freeze.

azure-applicationinsights==0.1.0
azure-batch==4.1.3
azure-common==1.1.16
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.5
azure-datalake-store==0.0.39
azure-eventgrid==1.2.0
azure-graphrbac==0.40.0
azure-keyvault==1.1.0
azure-loganalytics==0.1.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.50.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==2.0.0
azure-mgmt-billing==0.2.0
azure-mgmt-cdn==3.0.0
azure-mgmt-cognitiveservices==3.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==4.3.1
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==1.4.0
azure-mgmt-containerregistry==2.4.0
azure-mgmt-containerservice==4.2.2
azure-mgmt-cosmosdb==0.4.1
azure-mgmt-datafactory==0.6.0
azure-mgmt-datalake-analytics==0.6.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-datamigration==1.0.0
azure-mgmt-devspaces==0.1.0
azure-mgmt-devtestlabs==2.2.0
azure-mgmt-dns==2.1.0
azure-mgmt-eventgrid==1.0.0
azure-mgmt-eventhub==2.2.0
azure-mgmt-hanaonazure==0.1.1
azure-mgmt-iotcentral==0.1.0
azure-mgmt-iothub==0.5.0
azure-mgmt-iothubprovisioningservices==0.2.0
azure-mgmt-keyvault==1.1.0
azure-mgmt-loganalytics==0.2.0
azure-mgmt-logic==3.0.0
azure-mgmt-machinelearningcompute==0.4.1
azure-mgmt-managementgroups==0.1.0
azure-mgmt-managementpartner==0.1.0
azure-mgmt-maps==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==1.0.0
azure-mgmt-monitor==0.5.2
azure-mgmt-msi==0.2.0
azure-mgmt-network==2.3.0
azure-mgmt-notificationhubs==2.0.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-policyinsights==0.1.0
azure-mgmt-powerbiembedded==2.0.0
azure-mgmt-rdbms==1.5.0
azure-mgmt-recoveryservices==0.3.0
azure-mgmt-recoveryservicesbackup==0.3.0
azure-mgmt-redis==5.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.2.1
azure-mgmt-resource==2.0.0
azure-mgmt-scheduler==2.0.0
azure-mgmt-search==2.0.0
azure-mgmt-servicebus==0.5.3
azure-mgmt-servicefabric==0.2.0
azure-mgmt-signalr==0.1.1
azure-mgmt-sql==0.9.1
azure-mgmt-storage==2.0.0
azure-mgmt-subscription==0.2.0
azure-mgmt-trafficmanager==0.50.0
azure-mgmt-web==0.35.0
azure-mgmt==4.0.0
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-servicefabric==6.3.0.0
azure-servicemanagement-legacy==0.20.6
azure-storage-blob==1.4.0
azure-storage-common==1.4.0
azure-storage-file==1.4.0
azure-storage-queue==1.4.0
azure==4.0.0

What problem was encountered?

The docs for put_block and other put_foo methods describe the need for a valid base64 string, however this is incorrect:

1. The block service applies base64 encoding, and the raw value should be passed instead
2. The raw value must be selected only of ascii characters

Failure to observe the above results in "One of the request inputs is out of range." (which by the way, is a super unhelpful and unactionable error)

Traceback (most recent call last):
  File "cloud-transfer.py", line 221, in <module>
    upload(path, url)
  File "cloud-transfer.py", line 196, in upload
    handlers[scheme](path, url)
  File "cloud-transfer.py", line 137, in upload_azure_blob
    pycosio.copyfile(path, url)
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/pycosio/_core/functions_shutil.py", line 142, in copyfile
    _copy(src, dst, src_is_storage, dst_is_storage)
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/pycosio/_core/functions_shutil.py", line 71, in _copy
    copyfileobj(fsrc, fdst, buffer_size)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 119, in __exit__
    next(self.gen)
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/pycosio/_core/functions_io.py", line 79, in cos_open
    yield wrapped
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/pycosio/_core/io_buffered.py", line 119, in close
    self._close_writable()
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/pycosio/storage/azure_blob.py", line 385, in _close_writable
    future.result()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/blob/blockblobservice.py", line 186, in put_block
    timeout=timeout
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/blob/blockblobservice.py", line 1143, in _put_block
    self._perform_request(request)
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/common/storageclient.py", line 430, in _perform_request
    raise ex
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/common/storageclient.py", line 358, in _perform_request
    raise ex
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/common/storageclient.py", line 344, in _perform_request
    HTTPError(response.status, response.message, response.headers, response.body))
  File "/Users/stewartadam/devel/virtualenvs/filetransfer-2tnmA_nJ/lib/python3.7/site-packages/azure/storage/common/_error.py", line 115, in _http_error_handler
    raise ex
azure.common.AzureHttpError: One of the request inputs is out of range. ErrorCode: OutOfRangeInput
<?xml version="1.0" encoding="utf-8"?><Error><Code>OutOfRangeInput</Code><Message>One of the request inputs is out of range.
RequestId:cbc31948-901e-001a-2ae7-82322b000000
Time:2018-11-23T04:45:08.4785074Z</Message></Error>

Have you found a mitigation/solution?

1. Update the docs
2. Pass the raw block IDs using ''.join(random.choice(string.ascii_lowercase) for i in range(length)) as shown here

[rickle-msft]

Hi, @stewartadam. Thank you for pointing this out to us. We will discuss on our team the best way to clarify this information an publish an update as soon as we can! The fix should be available by 7/12/2019, close the issue currently, feel free to reopen it if you have further questions!