Provide a way to adjust the httpclient timeout for layer pushes

baronfel commented 5 months ago

Users aren't always in control of their network - we should provide some way for them to override the default .NET timeout for uploads (100s). This pops up in a few places, like https://github.com/Azure/azure-dev/discussions/3212.

Thoughts:

since the user can cancel the operation at any time, should we just set an infinite timeout? If we do, we will likely need to adapt all existing callsites for the small operations to have short timeouts to prevent hangs. The Timeout in the HttpClient is used to create a linked cancellationToken that cancels after that duration, so we can orchestrate the same here - or simply use the new WaitAsync(TimeSpan) methods on Task<T> to make that super clear.
- Note that if we use the generic WaitAsync, those throw TimeoutExceptions with little detail, so we should handle them and make them say something useful.

cc @rainersigwald @MichalPavlik

BartNetJS commented 5 months ago

got the same issue, see https://github.com/Azure/azure-dev/issues/3225#top

baronfel commented 4 months ago

This also bit @mitchdenny hard today.

baronfel commented 4 months ago

Customers are hitting this repeatedly on the linked issue. We may need to bump priority + backport to 8.0.200 since Aspire is using that feature branch.

cc @MichalPavlik / @rokonec / @rainersigwald