Azure SDK failed to write request body issue

[nawab-alam]

Query/Question Our application using Azure SDK C++ library to perform upload blocks on block blob. Application perform upload on different blobs in parallel on Azurite emulator. Each blob may contain multiple blocks. Blocks uploaded using asynchronous version and periodically application checks pplx task whether upload successfully or not. We observed that intermittently application receive exception from pplx task get() API. We confirmed there is no any PUT request received at Azurite side when application get write failed error. On error application needs to restart complete upload operation.

Exception error as follows -

Failed to write request body.

Questions as below -

1. Is this known issue?
2. Here application do not have any control on these errors.     - Does Azure SDK provide ways to retry such failed task?     - How application should handle such issues?

Below is code snippet for reference.

• API used to Initialize context. There is pool of AzureClient object. Every upload operation get client from pool. If no client available then new client created.

  class AzureClient { azure::storage::cloud_storage_account _storageAccount; azure::storage::cloud_blob_client _blobClient; azure::storage::cloud_blob_container _container; azure::storage:: cloud_block_blob _cloudBlob; }

1. //Create storage account object _storageAccount = new azure::storage::cloud_storage_account(azure::storage::cloud_storage_account::parse(utility::string_t(connStr)));

2. //Create blob client object _blobClient = new azure::storage::cloud_blob_client(_storageAccount->blob_endpoint(),_storageAccount->credentials());

3. //Retrieve container reference using connection string. _container = new azure::storage::cloud_blob_container(utility::string_t(_containerName), *_blobClient)

4. //If container exist then create blob reference _cloudBlob = new azure::storage::cloud_block_blob(cloudBlob);

• API used to upload content to blob using Asynchronously

1. std::string azureData(data, blockSize);
2. concurrency::streams::istream inputStream = concurrency::streams::bytestream::open_istream(utility::conversions::to_utf8string(_XPLATSTR(azureData)));
3. pplx::task task = _cloudBlob->upload_block_async(utility::conversions::to_base64(_blockCount), inputStream, utility::string_t());
4. _writeTasks->push_back(task);

• API used to check completed task

  for (auto writeTask = _writeTasks->begin(); writeTask != _writeTasks->end(); ++writeTask) { writeTask->task.get(); }

Why is this not a Bug or a feature Request? This seem appropriate template for my issue. I am not looking for additional enhancement so it is not feature request. I am not sure whether it is Bug or not.

Setup (please complete the following information if applicable):

• OS: Linux
• IDE : n/a

• Version of the Library used     - Azure Storage Client Library for C++ 7.5.0     - cpprestsdk 2.10.18

  Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • [x] Query Added
  • [x] Setup information Added

[Jinming-Hu]

Is this known issue?

No, we've never seen this issue before.

Here application do not have any control on these errors.     - Does Azure SDK provide ways to retry such failed task?

Yes. Every API takes a blob_request_options as parameter, which has a member function of set_retry_policy.

    - How application should handle such issues?

We are still unclear about how this issue happened. It'd be too early to say how to handle it. We need some more information to further investigate.

• What's the typical HTTP request concurrency when this issue happened?
• What's the hardware like?
• How often does it happen? Like one out of how many requests failed with this exception?

[nawab-alam]

Thank you for your response. We are currently working on retry policy. Below are the answers of queries you asked for further investigation.

What's the typical HTTP request concurrency when this issue happened?

We were not sure about concurrency count. So it took time to gather required information. HTTP request concurrency is around 40.

What's the hardware like?

We are using shared hardware that consist of 144 processors and 1TB memory.

How often does it happen? Like one out of how many requests failed with this exception?

Issue is intermittent. We can consider failing frequency is hardly 1%-2%.