Flaky test: WebHookAction_NoEventHeader_IsNotFound

ryanbrandenburg commented 6 years ago

This test fails occasionally with the following error:

Microsoft.AspNetCore.Routing.Matchers.AmbiguousMatchException : The request matched multiple endpoints. Matches: 
GitHubCoreReceiver.Controllers.GitHubController.HandlerForItsPushes (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.HandlerForIt (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.HandlerForPush (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.GitHubHandler (GitHubCoreReceiver)
GitHubCoreReceiver.Controllers.GitHubController.FallbackHandler (GitHubCoreReceiver)
   at Microsoft.AspNetCore.Routing.EndpointConstraints.EndpointConstraintEndpointSelector.SelectAsync(HttpContext httpContext, IEndpointFeature feature, CandidateSet candidates)
   at Microsoft.AspNetCore.Routing.Matchers.DfaMatcher.MatchAsync(HttpContext httpContext, IEndpointFeature feature)
   at Microsoft.AspNetCore.Routing.GlobalRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.TestHost.HttpContextBuilder.<>c__DisplayClass10_0.<<SendAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.TestHost.ClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Testing.Handlers.CookieContainerHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Testing.Handlers.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.AspNetCore.WebHooks.FunctionalTest.GitHubCoreReceiverTest.WebHookAction_NoEventHeader_IsNotFound() in /_/test/Microsoft.AspNetCore.WebHooks.FunctionalTest/GitHubCoreReceiverTest.cs:line 42
--- End of stack trace from previous location where exception was thrown ---

ryanbrandenburg commented 6 years ago

WebHookAction_NoEventHeader_IsNotFound failed again.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

ryanbrandenburg commented 6 years ago

WebHookAction_NoEventHeader_IsNotFound failed again.

Eilon commented 6 years ago

@dougbu - Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- [x] File a bug against the product (#318, now aspnet/Routing#630)
- Is it possible to change the test to avoid the flakiness?
- Yes? Go to the "Change the test!" section.
- No?
  - [x] Disable the test (PR #320, commit bedc118aef -- really a workaround)
  - [ ] Wait for other bug to be resolved
  - [ ] Wait for us to get build that has the fix
  - [ ] Re-enable our test (TODO: Link to PR/commit)
  - [ ] Close this bug
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
- Yes? Change the test!
  - [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
  - [ ] Run the test 100 times locally as a sanity check.
  - [ ] Close this bug
- No?
  - [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

dougbu commented 6 years ago

See #318 for product bug

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

Eilon commented 6 years ago

@dougbu - please update the "TODO" items in the workflow to reflect the status.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NoEventHeader_IsNotFound failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NonPost_IsNotAllowed failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NoEventHeader_IsNotFound failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

ryanbrandenburg commented 6 years ago

WebHookAction_NoEventHeader_IsNotFound failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

dougbu commented 6 years ago

@rynowak much appreciated.

Is this done? And, should the workflow above be updated?

aspnet / WebHooks

Flaky test: WebHookAction_NoEventHeader_IsNotFound #315