Flaky test: IdentityUI_ScriptTags_SubresourceIntegrityCheck

[ryanbrandenburg]

This test fails occasionally with the following error:

System.Net.Http.HttpRequestException : No such host is known.
---- System.Net.Sockets.SocketException : No such host is known.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.Http.HttpClient.FinishGetStreamAsync(Task`1 getTask)
   at Microsoft.AspNetCore.Identity.Test.IdentityUIScriptsTest.IdentityUI_ScriptTags_SubresourceIntegrityCheck(ScriptTag scriptTag) in /_/test/Identity.Test/IdentityUIScriptsTest.cs:line 44
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

CC @Eilon,@muratg,@mkArtakMSFT

This issue was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

• Is this actually a flaky test?

  • No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
  • Yes, proceed below...

• Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

  • [ ] File a bug against the product (TODO: Link to other bug)
  • Is it possible to change the test to avoid the flakiness?
  • Yes? Go to the "Change the test!" section.
  • No?
    • [ ] Disable the test (TODO: Link to PR/commit)
    • [ ] Wait for other bug to be resolved
    • [ ] Wait for us to get build that has the fix
    • [ ] Re-enable our test (TODO: Link to PR/commit)
    • [ ] Close this bug

• Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

  • Is there is a way to change our test to avoid this flakiness?
  • Yes? Change the test!
    • [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    • [ ] Run the test 100 times locally as a sanity check.
    • [ ] Close this bug
  • No?
    • [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

IdentityUI_ScriptTags_SubresourceIntegrityCheck failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[Eilon]

@javiercn - please take a look at this high priority flaky test issue.

[javiercn]

@kichalla As I believe he wrote these tests :)

[kichalla]

IdentityUI_ScriptTags_SubresourceIntegrityCheck failed again.

'failed again' seems incorrect here...both the OP and the new comment point to the same build..

[kichalla]

Not a flaky test actually. Looking at test history, it has been very stable. The failure mentioned in this bug is a one off thing I would say. We could add retry logic as a general good practice, but I wouldn't consider it as a high priority.

[Eilon]

What does "one off" mean? Can it not fail again in the same way? Was there a bug that was fixed?

[kichalla]

This test failed because it was unable to reach ajax.aspnetcdn.com. Sure, there's a possibility that it could fail again for similar reasons. I was thinking of adding a retry logic to these tests, but was thinking that could be done at a lower priority considering the test's history. (Even if we add retry logic, its still not fail proof though). No, there was no bug that was fixed.

[Eilon]

Ah OK then we can keep this closed because we were having general network issues at around that time.