Test failure: CanLogInAfterConfirmingEmail

ryanbrandenburg commented 6 years ago

This test fails occasionally with the following error:

System.IO.IOException : Too many open files
   at Interop.Sys.GetCwdHelper(Byte* ptr, Int32 bufferSize)
   at Interop.Sys.GetCwd()
   at System.Environment.get_CurrentDirectory()
   at System.IO.Directory.GetCurrentDirectory()
   at Identity.DefaultUI.WebSite.Program.CreateWebHostBuilder(String[] args)

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

This test failed on master.

CC @Eilon (because the bot doesn't know who else to pick)

This issue was made automatically. If there is a problem contact ryanbrandenburg.

ryanbrandenburg commented 6 years ago

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this actually a flaky test?
- No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
- Yes, proceed below...
Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- [ ] File a bug against the product (TODO: Link to other bug)
- Is it possible to change the test to avoid the flakiness?
- Yes? Go to the "Change the test!" section.
- No?
  - [ ] Disable the test (TODO: Link to PR/commit)
  - [ ] Wait for other bug to be resolved
  - [ ] Wait for us to get build that has the fix
  - [ ] Re-enable our test (TODO: Link to PR/commit)
  - [ ] Close this bug
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
- Yes? Change the test!
  - [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
  - [ ] Run the test 100 times locally as a sanity check.
  - [ ] Close this bug
- No?
  - Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
  - Yes?
    - [ ] Add the logging (TODO: Link to PR/commit)
  - No?
    - [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ryanbrandenburg commented 6 years ago

CanSetPasswordWithExternalLogin failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ryanbrandenburg commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ryanbrandenburg commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

ryanbrandenburg commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

aspnet-hello commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInWithTwoFactorAuthentication_WithGlobalAuthorizeFilter failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

aspnet-hello commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on 2.2. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

aspnet-hello commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Eilon commented 6 years ago

@HaoK can you do an initial investigation here? Seems unlikely to be Identity-specific, so we need to figure out what the real issue is here.

HaoK commented 6 years ago

I noticed that there was some mention of the "beefy" ness of the test machines has changed recently? These tests have been very stable in the past, and only recently have started failing with these out of memory/too many file open type errors. Have we seen any other similar failures due to the capability of the tests machines being less?

aspnet-hello commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

Eilon commented 6 years ago

@natemcmaster - do we know when we might get beefier VSTS agents?

natemcmaster commented 6 years ago

Next week. :)

aspnet-hello commented 6 years ago

CanLogInAfterConfirmingEmail failed with about the same error on master. CanSetPasswordWithExternalLogin failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

aspnet-hello commented 6 years ago

CanSetPasswordWithExternalLogin failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master. CanLogInAfterConfirmingEmail failed with about the same error on master.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

HaoK commented 6 years ago

@Eilon should I resolve this optimistically since this seems similar to https://github.com/aspnet/Identity/commit/64af0228d60958cc579b6f323e1912fccf18f61c maybe it was the parallelization that was making things freak out?

Eilon commented 6 years ago

Go for it. If it's not still fixed the bot will do the needful.

HaoK commented 6 years ago

Hopefully fixed as well by https://github.com/aspnet/Identity/pull/1944

aspnet / Identity

Test failure: CanLogInAfterConfirmingEmail #1923