Test failure: QueryingIdentityEndpointsReturnsNotFoundWhenIdentityIsNotPresent

[aspnet-hello]

This test fails occasionally with the following error:

System.InvalidOperationException : Can't find'C:\b\w\33bdfc1cae7b2a38\modules\Identity\test\Identity.FunctionalTests\bin\Release\netcoreapp2.2\Identity.DefaultUI.WebSite.deps.json'. This file is required for functional tests to run properly. There should be a copy of the file on your source project bin folder. If that is not the case, make sure that the property PreserveCompilationContext is set to true on your project file. E.g '<PreserveCompilationContext>true</PreserveCompilationContext>'. For functional tests to work they need to either run from the build output folder or the Identity.DefaultUI.WebSite.deps.json file from your application's output directory must be copied to the folder where the tests are running on. A common cause for this error is having shadow copying enabled when the tests run.
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureDepsFile()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
   at Microsoft.AspNetCore.Identity.FunctionalTests.NoIdentityAddedTests.QueryingIdentityEndpointsReturnsNotFoundWhenIdentityIsNotPresent(String endpoint) in /_/test/Identity.FunctionalTests/NoIdentityAddedTests.cs:line 31
--- End of stack trace from previous location where exception was thrown ---

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

This test failed on 2.2.

CC @Eilon (because the bot doesn't know who else to pick)

This issue was made automatically. If there is a problem contact ryanbrandenburg.

[aspnet-hello]

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

• Is this actually a flaky test?

  • No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
  • Yes, proceed below...

• Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

  • [ ] File a bug against the product (TODO: Link to other bug)
  • Is it possible to change the test to avoid the flakiness?
  • Yes? Go to the "Change the test!" section.
  • No?
    • [ ] Disable the test (TODO: Link to PR/commit)
    • [ ] Wait for other bug to be resolved
    • [ ] Wait for us to get build that has the fix
    • [ ] Re-enable our test (TODO: Link to PR/commit)
    • [ ] Close this bug

• Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

  • Is there is a way to change our test to avoid this flakiness?
  • Yes? Change the test!
    • [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    • [ ] Run the test 100 times locally as a sanity check.
    • [ ] Close this bug
  • No?
    • Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
    • Yes?
      • [ ] Add the logging (TODO: Link to PR/commit)
    • No?
      • [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

[aspnet-hello]

AnonymousUserCantAccessAuthorizedPages failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

[Eilon]

@mkArtakMSFT - please assign.

[aspnet-hello]

QueryingIdentityEndpointsReturnsNotFoundWhenIdentityIsNotPresent failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

[ryanbrandenburg]

@mkArtakMSFT was this resolved? If so how? Please assign someone to find out.

[Eilon]

@javiercn ? @pranavkm ?

[javiercn]

@Eilon Sounds like an MSBuild issue/race condition?

We haven't seen this on MVC and it's not a problem customers have reported.

[Eilon]

Can we dig into that a bit more? Are we assuming determinism of MSBuild when perhaps that is an invalid assumption? Can we get in touch with those folks to try to understand this better?

[pranavkm]

Could we close this? No reports of additional failures over 2 weeks now.

[ryanbrandenburg]

Do you have a hypothesis on why this might have started passing again?

[javiercn]

@ryanbrandenburg I think this has been a transient issue. Our stack is so complex that I wouldn't even know where to start in order to look into this. I'm confident that this is not something we have caused as we haven't changed this code in months, so I will agree with @pranavkm in that we should close this.

The only thing we can do is to enable binary logs in all our builds so that we can do postmortems on these things, but without that, it's impossible to troubleshoot.

[javiercn]

I don't see this issue anymore, in any build for the past month, so closing.