Flaky test: Server_SetConnectionLimit_Success

[ryanbrandenburg]

This test fails occasionally with the following error:

System.Net.Http.HttpRequestException : An error occurred while sending the request.
---- System.IO.IOException : Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
-------- System.Net.Sockets.SocketException : An existing connection was forcibly closed by the remote host.
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.Http.HttpClient.GetStringAsyncCore(Task`1 getTask)
   at Microsoft.AspNetCore.Server.HttpSys.ServerTests.SendRequestAsync(String uri) in /_/test/Microsoft.AspNetCore.Server.HttpSys.FunctionalTests/ServerTests.cs:line 638
   at Microsoft.AspNetCore.Server.HttpSys.ServerTests.Server_SetConnectionLimit_Success() in /_/test/Microsoft.AspNetCore.Server.HttpSys.FunctionalTests/ServerTests.cs:line 378
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
----- Inner Stack Trace -----

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

CC @Eilon,@muratg,@mkArtakMSFT

This issue was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

• Is this actually a flaky test?

  • No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
  • Yes, proceed below...

• Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

  • [ ] File a bug against the product (TODO: Link to other bug)
  • Is it possible to change the test to avoid the flakiness?
  • Yes? Go to the "Change the test!" section.
  • No?
    • [ ] Disable the test (TODO: Link to PR/commit)
    • [ ] Wait for other bug to be resolved
    • [ ] Wait for us to get build that has the fix
    • [ ] Re-enable our test (TODO: Link to PR/commit)
    • [ ] Close this bug

• Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

  • Is there is a way to change our test to avoid this flakiness?
  • Yes? Change the test!
    • [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    • [ ] Run the test 100 times locally as a sanity check.
    • [ ] Close this bug
  • No?
    • [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

@muratg was this resolved? If so how? Please assign someone to find out.

[muratg]

@Tratcher

[Tratcher]

No changes were made. This was the only report of this failure in recorded history. The test failed two other times in the last year with a different error.

Assert.Throws() Failure
Expected: typeof(System.Net.Http.HttpRequestException)
Actual:   (No exception was thrown)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.AspNetCore.Server.HttpSys.ServerTests.<Server_SetConnectionLimit_Success>d__13.MoveNext() in C:\b\w\33bdfc1cae7b2a38\modules\HttpSysServer\test\Microsoft.AspNetCore.Server.HttpSys.FunctionalTests\ServerTests.cs:line 335
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

[Eilon]

@Tratcher can we add more logging to this case so that we can find out more next time it fails? Ultimately the concern is that there could be a lingering product bug.

[Tratcher]

I did some code cleanup that should help here.