Test failure: ShutdownTestRun

[ryanbrandenburg]

This test fails occasionally with the following error:

System.NullReferenceException : Object reference not set to an instance of an object.
   at Microsoft.AspNetCore.Server.IntegrationTesting.ApplicationDeployer.CleanPublishedOutput() in /_/src/Microsoft.AspNetCore.Server.IntegrationTesting/Deployers/ApplicationDeployer.cs:line 100
   at Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer.Dispose() in /_/src/Microsoft.AspNetCore.Server.IntegrationTesting/Deployers/SelfHostDeployer.cs:line 205
   at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ExecuteShutdownTest(String testName, String shutdownMechanic) in /_/test/Microsoft.AspNetCore.Hosting.FunctionalTests/ShutdownTests.cs:line 114
   at Microsoft.AspNetCore.Hosting.FunctionalTests.ShutdownTests.ShutdownTestRun() in /_/test/Microsoft.AspNetCore.Hosting.FunctionalTests/ShutdownTests.cs:line 33
--- End of stack trace from previous location where exception was thrown ---
------- Stdout: -------
| [0.015s] TestLifetime Information: Starting test ShutdownTestRun at 2018-08-10T22:40:24
| [0.001s] TestLifetime Information: Starting test ExecuteShutdownTest at 2018-08-10T22:40:24
| [0.009s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: Deploying [Variation] :: ServerType=Kestrel, Runtime=CoreClr, Arch=x64, BaseUrlHint=, Publish=True
| [0.014s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: Executing command dotnet publish  --output "/mnt/temp/buildTmp/27932f56ea2345cc9813f135d51da328" --framework netcoreapp2.2 --configuration Release --no-restore -p:VerifyMatchingImplicitPackageVersion=false 
| [0.332s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: dotnet-publish stdout: Microsoft (R) Build Engine version 15.7.179.6572 for .NET Core
| [0.333s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: dotnet-publish stdout: Copyright (C) Microsoft Corporation. All rights reserved.
| [2.505s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: dotnet-publish stdout: /mnt/work/33bdfc1cae7b2a38/.packages/internal.aspnetcore.sdk/2.2.0-preview1-20180807.2/build/GenerateAssemblyInfo.targets(25,5): error MSB4062: The "Microsoft.AspNetCore.BuildTools.Sdk_GetAssemblyFileVersion" task could not be loaded from the assembly /mnt/work/33bdfc1cae7b2a38/.packages/internal.aspnetcore.sdk/2.2.0-preview1-20180807.2/build/../tools/netcoreapp2.1/Internal.AspNetCore.Sdk.dll. Assembly with same name is already loaded Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/mnt/work/33bdfc1cae7b2a38/modules/Hosting/src/Microsoft.Extensions.Hosting.Abstractions/Microsoft.Extensions.Hosting.Abstractions.csproj]
| [2.515s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Information: dotnet-publish stdout: /mnt/work/33bdfc1cae7b2a38/.packages/internal.aspnetcore.sdk/2.2.0-preview1-20180807.2/build/GenerateAssemblyInfo.targets(25,5): error MSB4062: The "Microsoft.AspNetCore.BuildTools.Sdk_GetAssemblyFileVersion" task could not be loaded from the assembly /mnt/work/33bdfc1cae7b2a38/.packages/internal.aspnetcore.sdk/2.2.0-preview1-20180807.2/build/../tools/netcoreapp2.1/Internal.AspNetCore.Sdk.dll. Assembly with same name is already loaded Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/mnt/work/33bdfc1cae7b2a38/modules/Hosting/src/Microsoft.AspNetCore.Hosting.Server.Abstractions/Microsoft.AspNetCore.Hosting.Server.Abstractions.csproj]
| [2.602s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Error: dotnet publish exited with exit code : 1
| [2.605s] Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer Warning: Host process already exited or never started successfully.
| [2.607s] TestLifetime Information: Finished test ExecuteShutdownTest in 2.6064648s
| [2.664s] TestLifetime Information: Finished test ShutdownTestRun in 2.6582765s

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

This test failed on 2.2.

CC @Eilon (because the bot doesn't know who else to pick)

This issue was made automatically. If there is a problem contact ryanbrandenburg.

[ryanbrandenburg]

ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on 2.2. ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on 2.2. ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on master. Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

• Is this actually a flaky test?

  • No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
  • Yes, proceed below...

• Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

  • [ ] File a bug against the product (TODO: Link to other bug)
  • Is it possible to change the test to avoid the flakiness?
  • Yes? Go to the "Change the test!" section.
  • No?
    • [ ] Disable the test (TODO: Link to PR/commit)
    • [ ] Wait for other bug to be resolved
    • [ ] Wait for us to get build that has the fix
    • [ ] Re-enable our test (TODO: Link to PR/commit)
    • [ ] Close this bug

• Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

  • Is there is a way to change our test to avoid this flakiness?
  • Yes? Change the test!
    • [ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
    • [ ] Run the test 100 times locally as a sanity check.
    • [ ] Close this bug
  • No?
    • Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
    • Yes?
      • [ ] Add the logging (TODO: Link to PR/commit)
    • No?
      • [ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

[Tratcher]

Duplicate of #1503, closed a little too soon @natemcmaster

[natemcmaster]

This bot is very aggressive about squashing bugs.

[ryanbrandenburg]

ShutdownTestRun failed with about the same error on 2.2. ShutdownTestRun failed with about the same error on master. ShutdownTestRun failed with about the same error on 2.2.

This comment was made automatically. If there is a problem contact ryanbrandenburg.

[muratg]

@Tratcher This one good to close yet?