search

Azure / azure-sdk-tools

Tools repository leveraged by the Azure SDK team.
MIT License
110 stars 174 forks source link

Requests failing with `Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.` #5673

Closed scbedd closed 1 year ago

scbedd commented 1 year ago

azure-keyvault-administration tests are failing unexpectedly with some very odd behavior.

When the failing tests are run individually, this behavior does not repro. Otherwise, it only repros when all tests are run for the package. (like in CI)

I personally think that there is something going a bit wonky with the keyvault-administration client. Caching or the like? Sending wrong content-length? But we need to dig in and find out.

In the meantime going to dash out a PR that adjust MinRequestBodyDataRate.

Stacktrace:

at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.ReadAsyncInternal(CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder\u00601.StateMachineBox\u00601.System.Threading.Tasks.Sources.IValueTaskSource\u003CTResult\u003E.GetResult(Int16 token)
at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func\u00604 writeAsync, CancellationToken cancellationToken)
at Azure.Sdk.Tools.TestProxy.RecordingHandler.ReadAllBytes(Stream stream) in /mnt/vss/_work/1/s/tools/test-proxy/Azure.Sdk.Tools.TestProxy/RecordingHandler.cs:line 1081
at Azure.Sdk.Tools.TestProxy.RecordingHandler.CreateEntryAsync(HttpRequest request) in /mnt/vss/_work/1/s/tools/test-proxy/Azure.Sdk.Tools.TestProxy/RecordingHandler.cs:line 547
at Azure.Sdk.Tools.TestProxy.RecordingHandler.HandlePlaybackRequest(String recordingId, HttpRequest incomingRequest, HttpResponse outgoingResponse) in /mnt/vss/_work/1/s/tools/test-proxy/Azure.Sdk.Tools.TestProxy/RecordingHandler.cs:line 438
at Azure.Sdk.Tools.TestProxy.Playback.HandleRequest() in /mnt/vss/_work/1/s/tools/test-proxy/Azure.Sdk.Tools.TestProxy/Playback.cs:line 88
at lambda_method45(Closure , Object )
at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.\u003CInvokeActionMethodAsync\u003Eg__Awaited|12_0(ControllerActionInvoker invoker, ValueTask\u00601 actionResultValueTask)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.\u003CInvokeNextActionFilterAsync\u003Eg__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask
scbedd commented 1 year ago

The was not on test-proxy side. The failure was because the request the python tests were sending had a Content-Length, but never actually sent a body.

Reason was that the test was caching an empty credential body erroneously. Ensuring the cache was properly cleared means that the body would be sent with actual value, which resolved the problem.