Running tests from VSCode test explorer ignores "envVars in IntegrationTest" from build.sbt

[martin-stone]

Describe the bug

In my build.sbt I have something like...

fork in IntegrationTest := true
envVars in IntegrationTest := Map("VAR_NAME" -> "value")

When running integration tests in the terminal with SBT, I can access the value in the test code with...

sys.env("VAR_NAME")

but when running the test from the VSCode GUI the variable is undefined.

Expected behavior

I would expect tests to function the same when run in an IDE as when run in the terminal and see the variables defined in build.sbt.

Operating system

Linux

Editor/Extension

VS Code

Version of Metals

0.11.12

Extra context or search terms

I'm running VSCode in Windows with a Debian devcontainer.

[tgodzik]

Thanks for reporting! Are you using the default Bloop or sbt via BSP? The latter option might work better in your case since Bloop doesn't have a way to store environment variables.

As a workaround you can specify the enn vars in the launch.json configuration

[martin-stone]

Are you using the default Bloop or sbt via BSP?

Looks like I've been using Bloop.

With SBT/BSP, I've not been able to launch my integration tests using launch.json. I get...

[Error - 07:10:28] Starting debug server failed, see metals.log for more details.
[Error - 07:10:28] Request workspace/executeCommand failed.
  Message: Internal error.
  Code: -32603 
java.util.concurrent.CompletionException: scala.meta.internal.metals.MetalsBspException: BSP connection failed in the attempt to get: DebugSessionAddress
...

(Full log output: stack.txt )

Running unit tests or a "mainClass" works as expected.

Ideally I'd have a setup where the rest of my team can simply click "run tests" in the test explorer and it works as expected. The environment vars in build.sbt are somewhat dynamic (e.g. automatically finding the latest version of a dependency), and manually configuring a duplicate in launch.json seems like a problem waiting to happen.

[tgodzik]

Sure! We need to take a look into an issue with running integration tests. As for env variables, I think currently we can make it work only in sbt (after fixing). Bloop might be harder since you would reimport the build every time the env var changes.

Another option for sbt would be to use the BuildInfo plugin, which just puts the build information into generated sources to be used directly in the code. I find that a bit better then env vars since we can be sure that everything is set up properly if it compiles. Of course I don't exactly know your use case

[tgodzik]

As part of this issue let's check and fix the integration tests.

[kasiaMarek]

With sbt metals uses the debug adapter plugin for running tests. The plugin adds correct tasks for Test config, which are used to start a debug session (run tests). The tasks can't added to custom configurations (as IntergationTest) from the plugin level, since it isn't known what custom configurations are used.

@martin-stone you can add necessary setting to your IntergationTest to enable dap by adding

inConfig(IntegrationTest)(DebugAdapterPlugin.testSettings)

to the project definition in build.sbt. With that your tests should run and see envVars properly (at least with sbt build server).

Also note, that custom configurations have been deprecated in sbt.

[tgodzik]

I think the current solution for this will be implementing https://github.com/scalameta/metals/issues/5928

This will make it not necessary to add any special configuration for sbt. Since, that is a direction we want to follow I will close this one