Test explorer: "Run test" works while "Debug test" fails to build FSharp compiler tests

[Martin521]

Describe the bug

In the situation detailed in "Steps to reproduce", I can run any test from the test explorer by clicking Run Test. However, if I click Debug Test, I get the error message shown below. Note that in both cases, I see a notification Running MSBuild 'Build' target on '/workspaces/fsharp/tests/FSharp.Compiler.ComponentTests/FSharp.Compiler.ComponentTests.fsproj'.

Error message in the TEST RESULTS window:

/workspaces/fsharp/FSharpBuild.Directory.Build.props(3,3): error : Could not resolve SDK "Microsoft.DotNet.Arcade.Sdk". Exactly one of the probing messages below indicates why we could not resolve the SDK. Investigate and resolve that message to correctly specify the SDK.
/workspaces/fsharp/FSharpBuild.Directory.Build.props(3,3): error :   SDK resolver "Microsoft.DotNet.MSBuildWorkloadSdkResolver" returned null.
/workspaces/fsharp/FSharpBuild.Directory.Build.props(3,3): error :   Failed to load NuGet settings. Required environment variable 'HOME' is not set. Try setting 'DOTNET_CLI_HOME' or 'HOME' and running the operation
/workspaces/fsharp/FSharpBuild.Directory.Build.props(3,3): error :   MSB4276: The default SDK resolver failed to resolve SDK "Microsoft.DotNet.Arcade.Sdk" because directory "/usr/share/dotnet/sdk/8.0.100-rc.1.23463.5/Sdks/Microsoft.DotNet.Arcade.Sdk/Sdk" did not exist.

/workspaces/fsharp/FSharpBuild.Directory.Build.props(3,31): error MSB4236: The SDK 'Microsoft.DotNet.Arcade.Sdk' specified could not be found. [/workspaces/fsharp/tests/FSharp.Compiler.ComponentTests/FSharp.Compiler.ComponentTests.fsproj]

Steps to reproduce

1. Run Dev Containers: Clone Repository in Named Container volume ... and clone the dotnet/fsharp repository.
2. Wait for the dev container to build and the test explorer to discover the tests
3. Run any test in FSharp.Compiler.ComponentTests (=> succeeds)
4. Debug a test (=> above error message)

Link to sample reproduction

FSharp

Expected behaviour

See the test running in the debugger

Machine info

• OS: Windows 11 host with Docker; dev container as specified by the fsharp repository
• .NET SDK version: 8.0.100-rc.1.23463.5
• Ionide version: 7.13.0

Additional context

Work-around: create a .NET Core Test with debugger task that runs dotnet test. Run the task and attach the debugger.

[TheAngryByrd]

This is probably related to BUILD_USING_DOTNET. I'm guessing we're not passing along environment variables when we set up for debugging

cc @farlee2121

[TheAngryByrd]

@Martin521 can you try forcing BUILDING_USING_DOTNET to be true in the Directory.Build.props? It'll help us figure out if it's the root of the issue.

[farlee2121]

I'm still trying to get the fsharp repo building with the dotnet cli, but the env theory seems probable.

[Martin521]

BUILDING_USING_DOTNET is set to true in the Dockerfile, and I can verify it in the terminal. At some point I also tested setting it in Directory.Build.props.user and that did not help. I can try that again.

[Martin521]

I'm still trying to get the fsharp repo building with the dotnet cli, but the env theory seems probable.

You will probably know this, but just in case: Make sure you build FSharp.Compiler.Service.sln and not FSharp.sln

[Martin521]

Actually, with BUILDING_USING_DOTNET set in Directory.Build.props.user, the build succeeds, but then I get runtime initialization error that I have seen before, at

            let pathToArtifacts = Path.GetFullPath(Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "../../../.."))
            if Path.GetFileName(pathToArtifacts) <> "artifacts" then failwith "CompilerAssert did not find artifacts directory --- has the location changed????"

in FSharp.Test.Utilities/Utilities.fs. This comes from .../Debug/... missing in the artifacts path. If I remove one ../ in the above code, it runs fine. (But then test discovery and simple run do not work any more)

[farlee2121]

Troubles Running dotnet/fsharp

Good callout, but I am building with FSharp.Compiler.Service.sln. I made sure I had the latest VisualStudio Preview, the latest .NET 7, the correct preview version of .NET, and pulled a clean copy of the repo. I'm still getting the same behavior. It builds with arcade, but dotnet build .\FSharp.Compiler.Service.sln still throws

Microsoft.FSharp.Targets(331,9): error MSB6006: "dotnet.exe" exited with code 1.

It does this for 13 of the 19 projects. The referenced line (Microsoft.FSharp.Targets(331,9)) happens to be the Fsc invocation.

Env Experiment

Since the build props worked, that seems to confirm issues with environment variables not getting passed along.

While I currently can't test against the fsharp repo, I did draft a change you can test out.

Initialization Error

As for the initialization issue, it'd be worth checking the Assembly.GetExecutingAssembly(). Sounds like it might change when debugging. You might be able to get away with using preprocessor directives.

[Martin521]

Thanks a lot. Unfortunately, I cannot test the draft right now as I don't have the necessary environment to build Ionide.

As for the initialization error, I assume it is also related to the environment variables.

[vzarytovskii]

Troubles Running dotnet/fsharp

Good callout, but I am building with FSharp.Compiler.Service.sln.

I made sure I had the latest VisualStudio Preview, the latest .NET 7, the correct preview version of .NET, and pulled a clean copy of the repo.

I'm still getting the same behavior. It builds with arcade, but dotnet build .\FSharp.Compiler.Service.sln still throws

Microsoft.FSharp.Targets(331,9): error MSB6006: "dotnet.exe" exited with code 1.

It does this for 13 of the 19 projects. The referenced line (Microsoft.FSharp.Targets(331,9)) happens to be the Fsc invocation.

Env Experiment

Since the build props worked, that seems to confirm issues with environment variables not getting passed along.

While I currently can't test against the fsharp repo, I did draft a change you can test out.

Initialization Error

As for the initialization issue, it'd be worth checking the Assembly.GetExecutingAssembly(). Sounds like it might change when debugging. You might be able to get away with using preprocessor directives.

You can either pass /bl switch which will generate binlog, or /v:n which will produce detailed logs

[farlee2121]

Thanks @vzarytovskii. That got me a more detailed error. I'll have to dig when I get back from my trip.

[farlee2121]

I got the repo building and running tests via dotnet test, but the test explorer doesn't pick up any of the tests for me. It doesn't think any of the projects are test projects because it doesn't seem to think any of them reference Microsoft.TestPlatform.TestHost or Microsoft.NET.Test.Sdk. Trying to explicitly add those packages references results in a duplicate reference build error. I'm not sure why it would work for @Martin521 and not for me...

[Martin521]

I have worked with the repo only in Dev Container, and there everything works like a breeze (including test discovery of the Test Explorer and running tests from the Test Explorer), except for debugging tests out of the Test Explorer.

[Martin521]

If you follow the "Steps to reproduce" mentioned in the issue text (which takes only a few minutes), you should be able to see it.

[farlee2121]

I suppose I didn't truly follow the reproduction steps. I wouldn't think the environment should affect the perceived packaged references this way, but the evidence seems to say otherwise...

Alas, while I've got the fsharp devcontainer running, it looks like the vscode debugging doesn't want to play nice with running against a dev container. You might need to get the Ionide repo running inside the dev container to further diagnose the issue / see if the env forwarding fixes the issue.

I did otherwise verify that accessing env variables behaves differently when running versus debugging. Seems like a bug in any case. The draft change does fix this issue, so I'm going to PR it.

[Martin521]

Thanks!