Not able to set a Client side timeout on requests to Blob storage

[neilrees]

The class BlobRequestOptions has two properties controlling timeouts:

• MaximumExecutionTime - Gets or sets the maximum execution time across all potential retries for the request.
• ServerTimeout - Gets or sets the server timeout interval for a single HTTP request.

When either uploading or downloading a blob that requires multiple HTTP requests to be sent, the timeout on each HttpWebRequest is set to be the remaining time before the whole operation expires rather than than the server timeout timespan: https://github.com/Azure/azure-storage-net/blob/v8.2.1/Lib/ClassLibraryCommon/Core/Executor/Executor.cs#L644

When downloading large blobs, e.g. TB in size, we want MaximumExecutionTime to be large, in hours, however if a single request is lost, the process waits until MaximumExecutionTime expires before timing out, as the server timeout (which maps through to a query string parameter) requires a response from the server.

If MaximumExecutionTime is set to a small value, so we timeout on a block sooner, and an IExtendedRetryPolicy is used it still cannot retry as the overall operation extends beyond the maximum allowed time: https://github.com/Azure/azure-storage-net/blob/v8.2.1/Lib/ClassLibraryCommon/Core/Executor/Executor.cs#L813

TLDR; Would like to be able to set a shorter timeout on individual blocks when uploading downloading so can retry a block sooner rather than waiting for MaximumExecutionTime to expire which may be large, and then having to retry the whole operation.

[pemari-msft]

Thanks for the submission. We are tracking addding this item in our 9.0 release. /cc: @kfarmer-msft @rickle-msft

[uhrb]

Guys, just moving this feature to Future does not solve the problem. Do you have any real dates when it will be resolved?

[krispenner]

I'd also like to see this addressed. I think a new timeout should be added, ClientTimeout which is the timeout of each single request (or retry).

Without testing the changes put forward by @neilrees, it looks like it would prevent the server from ever returning its own timeout if the request on the client is set to the same timeout as is sent to the server since you would have latency included on the request the client would always timeout first.

My proposal:

ServerTimeout = TimeSpan.FromSeconds(5); // sent to server to, as is
ClientTimeout = TimeSpan.FromSeconds(8);  // timeout of each individual request or retry attempt, ideally should be slightly larger than ServerTimeout to account for network latency
MaximumExecutionTime = TimeSpan.FromSeconds(30);  // timeout of all client retries, as is