ToolInstallCommandTests.WhenRunWithRoot is flaky

akoeplinger commented 3 months ago

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=753617 Build error leg or test failing: Microsoft.DotNet.Tests.Commands.Tool.ToolInstallCommandTests.WhenRunWithRoot Pull request: https://github.com/dotnet/sdk/pull/42343

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Microsoft.DotNet.Tests.Commands.Tool.ToolInstallCommandTests.WhenRunWithRoot [FAIL]",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=753617 Error message validated: [Microsoft.DotNet.Tests.Commands.Tool.ToolInstallCommandTests.WhenRunWithRoot [FAIL]] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 7/24/2024 4:34:26 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

akoeplinger commented 3 months ago

The test was added in https://github.com/dotnet/sdk/pull/40800 and it fails like this:

      StdErr:
      Creating this template will make changes to existing files:
        Overwrite   ./.config/dotnet-tools.json

      To create the template anyway, run the command with '--force' option:
         dotnet new tool-manifest --debug:custom-hive /tmp/folder --force

      For details on the exit code, refer to https://aka.ms/templating-exit-codes#73

      Stack Trace:
           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(String message)
        /_/test/Microsoft.NET.TestFramework/Assertions/CommandResultAssertions.cs(29,0): at Microsoft.NET.TestFramework.Assertions.CommandResultAssertions.Pass()
        /_/test/dotnet.Tests/CommandTests/ToolInstallCommandTests.cs(49,0): at Microsoft.DotNet.Tests.Commands.Tool.ToolInstallCommandTests.WhenRunWithRoot()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Looking at the test code I think this is because it uses Directory.SetCurrentDirectory and if other tests run at the same time this will mess up the working directory. https://github.com/dotnet/sdk/blob/753214aecd3ac7862960ce35d023675fe0717755/test/dotnet.Tests/CommandTests/ToolInstallCommandTests.cs#L39-L64

/cc @Forgind

nagilson commented 3 months ago

@akoeplinger Thank you for your investigation and filing this issue. I think your summarization looks correct. @Forgind, may you please look into fixing this?

dotnet / sdk