Add ability to set environment variables/system properties for test suites/individual tests

[mdedetrich]

Is your feature request related to a problem? Please describe.

In the latest version of Metals, support has been added for individual test lenses (see https://scalameta.org/metals/blog/#expose-test-cases-as-separate-lenses and https://github.com/scalameta/metals/pull/4569). For me personally this is one of the last few missing features to be able to almost completely replace Intellij with VSCode/Metals.

What is however missing is the ability to set environment variables and/or system properties for a test that you are about to execute. This is particularly useful when you have end to end/integration tests where you need to supply secrets (i.e. password to a database) in order for a test to work.

Describe the solution you'd like

I suspect that the solution would differ from editor to editor and its likely that metals only needs to add logic to accept a list of env vars/system properties for a specific test suite/lens and all of the UI work is done in the various editors supporting Metals/LSP. If true this would mean how the environment variables/system properties are specified and then stored would depend on editor to editor (for every case it should be on a project to project basis). Another place to store this data could also be in bloops local config for a project, in which case it comes with the advantage that these settings are then portable between different editors.

Ultimately I am looking for something akin to this in Intellij

Describe alternatives you've considered

Its obviously possible to work around this, its just more annoying/takes more work. One can always create a shell script that exports variables before a test is run but then you have to get the shell script to be integrated into the editor (you then also have the problem of having to remember not to commit this shell script because it contains secrets).

This isn't a hard blocker but the most ideal situation is what is being described.

Additional context

No response

Search terms

environment variables system properties test suite lens

[tgodzik]

Thanks for reporting! It's a bit problematic to create additional UI elements in all the editors, so maybe we could by default take in an env file for running tests such as test.env as a workaround?

Additionally, does this work if you specify the env variables inside the build tool? Such as: envVars in Test := Map("SCALA_ENV" -> "test")

for sbt? "test" can be replaced by getting an env variable. This should be used by the build tool later on when running, though I see that is a bit more complex, so maybe test.env workaround should be better :thinking:

[adpi2]

What about using a debug configuration to solve this?

I think it is already possible to have a list of env variables into the Scala Test configuration like this:

{
   "type": "scala",
   "request": "launch",
   "name": "Untitled",
   "testClass": "example.MyTestClass",
   "env": {}
},

So maybe what's missing is the ability to specify which test(s) to run?

"tests": ["test1", "test2"]

[mdedetrich]

Thanks for reporting! It's a bit problematic to create additional UI elements in all the editors, so maybe we could by default take in an env file for running tests such as test.env as a workaround?

Additionally, does this work if you specify the env variables inside the build tool? Such as: envVars in Test := Map("SCALA_ENV" -> "test")

for sbt? "test" can be replaced by getting an env variable. This should be used by the build tool later on when running, though I see that is a bit more complex, so maybe test.env workaround should be better :thinking:

Indeed these solutions as a workaround (and I am doing something similar, i.e. the envVars), just made this feature issue to discuss a more comprehensive solution.

To clarify one of the reasons behind a more comprehensive solution, one problem with envVars is that in the middle of everything you accidentally commit this addition in your build.sbt and suddenly you can accidentally push your secrets into github without thinking about it.

So maybe what's missing is the ability to specify which test(s) to run?

Not only tests suites but single test cases (latest metals has test lenses which lets you run individual test's within a suite).

[adpi2]

Not only tests suites but single test cases (latest metals has test lenses which lets you run individual test's within a suite).

Yes, that what I meant by "test(s)": individual tests.