search

vitest-dev / vscode

VS Code extension for Vitest
https://vitest.dev/vscode
MIT License
740 stars 83 forks source link

Vitest plugin fails to run tests in a monorepo with app-specific config #459

Closed anantharam closed 4 days ago

anantharam commented 3 weeks ago

Describe the bug

As discussed in https://github.com/vitest-dev/vscode/issues/446, the vscode extension fails tests when we have node configs that are not inside the root repository.

Reproduction

I have added a minimal reproduction here : https://stackblitz.com/edit/vitest-dev-vitest-hkrvhg.

Download the code from above repo, run npm install and then run npm test. This results the test passing successfully but when you run the same code inside the extension, the same test is failing.

You can see in this screenshot that the same test fails in vscode test and the test is passing when run from terminal : image

Output

[INFO 16:57:03] [v1.2.0] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[Error 16:57:03] [API] Vitest not found for file:///Users/ananthram/Downloads/Vitest%20Pnpm%20config/vite.config.ts.
[Error 16:57:03] [API] Failed to start Vitest: No vitest config files found
[INFO 16:58:39] [API] Running Vitest v2.0.5 (Vitest Pnpm config/vite.config.ts) with Node.js: /opt/homebrew/bin/node
[INFO 16:58:40] [API] Vitest v2.0.5 (Vitest Pnpm config/vite.config.ts) process 30353 created
[INFO 16:58:40] [API] Collecting tests: apps/math/src/test/math.test.ts
[INFO 16:58:40] [TEST] WARNING: NODE_ENV value of 'test' did not match any deployment config file names.
WARNING: See https://github.com/node-config/node-config/wiki/Strict-Mode
WARNING: No configurations found in configuration directory:/Users/ananthram/Downloads/Vitest Pnpm config/config
WARNING: To disable this warning set SUPPRESS_NO_CONFIG_WARNING in the environment.

[INFO 16:58:47] Running 1 file(s) with name pattern: ^\s?Server config works$
[INFO 17:02:08] Running 1 file(s): apps/math/src/test/math.test.ts
[INFO 17:42:19] Running 1 file(s): apps/math/src/test/math.test.ts
[INFO 17:43:07] [RUNNER] Watching test files: apps/math/src/test/math.test.ts with pattern ^\s?Server config works$

Version

1.2.0

Validations

sheremet-va commented 4 days ago

The reproduction doesn't work even when using the terminal. The intention of your package.json is not clear. You have several package.json configs, but the root one will give the same error in CLI and in the extension which is correct because the working directory is outside of math.

To run the tests correctly in your tests, you need to make apps/math a working directory and run vitest there. The extension cannot analyse this - since there is already a Vite config, it prioritises the config and doesn't crawl all package.json for performance reasons.

I would recommend creating a vitest.workspace.ts that lists every Vitest project. You can then run the tests from the root both in CLI and in the extension.

Alternatively, you can create a simple vitest.config.ts file in every project. The extension prioritises those.

anantharam commented 3 days ago

I added a minimum example with workspace and configs inside each app here : https://stackblitz.com/edit/vitest-workspace-example. This is closer to what we are using in our application, note that in the terminal we still run the pnpm test from inside the particular app (so inside the math folder in the example). This is like a monorepo with each app acting like a separate microservice. Also note that, this same exact setup works with https://marketplace.visualstudio.com/items?itemName=kingwl.vscode-vitest-runner extension and the tests run correctly but the tests fail with the official extension only. I assume it is because the other extension runs the tests inside the corresponding folder.

sheremet-va commented 3 days ago

This is a project-specific configuration issue. The extension works correctly by prioritizing your vitest.workspace.ts.

The config package that you rely on uses NODE_CONFIG_DIR environment variable which is ${process.cwd()}/config by default. When you run tests from the root, it uses the root working directory, and there is no config file there. To make this work with Vitest Workspace, you can define this variable in the config file of your projects, for example:

// apps/math/vitest.config.ts

import { fileURLToPath } from 'node:url';
import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    env: {
      NODE_CONFIG_DIR: fileURLToPath(new URL('./config', import.meta.url))
    },
  },
})

Make sure that config is not imported in that file, otherwise, it will fallback to the default variable.

Now you can run vitest from the root of the project (npm run vitest --project=math), and the extension also recognizes it correctly.

The other extension works differently and has a lot of assumptions that are not true in other cases. It just spawns vitest for specified files, we spawn vitest to collect and run files, and we don't respawn it again. This ensures a fast feedback.

anantharam commented 3 days ago

Thank you for the detailed explanation, it helps understanding why the extension is behaving this way.

I tried adding your suggested config and I have updated https://stackblitz.com/edit/vitest-workspace-example with that config inside apps/math/vitest.config.mts. You can now see that the test is passing in stackblitz but when I download it, the vscode extension is still failing. Also note that running npm run test --project=math from the root of the project still doesn't work. I am using test instead of your suggested vitest since the package.json only has script for test in this example. I am mentioning that since I assume it might be related.

image

sheremet-va commented 3 days ago

You can now see that the test is passing in stackblitz but when I download it, the vscode extension is still failing.

I cannot confirm this. The reproduction works fine locally with the extension.

Also note that running npm run test --project=math from the root of the project still doesn't work

Vitest uses package.json's name for project names by default. In your reproduction it would be vitest --project="@vitest/example-test".

anantharam commented 3 days ago

What I meant was that running npm run vitest --project=math gives npm error Missing script: "vitest" because there is no script called vitest, so I can only run npm run test and not npm run vitest. I now tried npm run test --project="@vitest/example-test" and that throws an error Error: Cannot find module '<dir>/Vitest workspace example/test'. You can see that in the terminal of the below screenshot.

I cannot confirm this. The reproduction works fine locally with the extension.

The steps would be to download https://stackblitz.com/edit/vitest-workspace-example, run npm install on the main folder and also run npm install inside the math folder. Then try to run the tests from inside the extension :

image

For info the output tab in vscode contains the following contents : 

[INFO 13:51:07] [v1.2.7] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[INFO 13:51:07] [API] Running Vitest v2.0.5 (Vitest workspace example/vitest.workspace.ts) with Node.js: /opt/homebrew/bin/node
[INFO 13:51:08] [API] Vitest v2.0.5 (Vitest workspace example/vitest.workspace.ts) process 43740 created
[INFO 13:51:14] Running 1 file(s): apps/math/src/test/math.test.ts
[13:51:14] The runner is starting because tests /Users/ananthram/Downloads/Vitest workspace example/apps/math/src/test/math.test.ts were started due to a file change
[13:51:14] Enqueuing "[@vitest/example-test] math.test.ts
[13:51:14] Marking "[@vitest/example-test] math.test.ts" as failed with 1 errors
[13:51:14] Marking "[@vitest/example-test] math.test.ts" as failed with 1 errors
[13:51:14] Marking "[@vitest/example-test] math.test.ts" as failed with 1 errors
[13:51:14] Ending test run apps/math/src/test/math.test.ts