Tests are being run with NODE_ENV=development

silvenon commented 5 months ago

Describe the bug

I had two problems when testing my Express server with supertest with the pre-release of the extension, this was one of them. I'm aware of process.env.VITEST_VSCODE, but that's just a workaround for this problem, NODE_ENV should still be set to test just like Vitest does.

I wasn't able to figure out the other problem, I keep getting "WebSocket server error: Port is already in use" in my logs and all of my server tests fail through the extension with 404 status instead of the expected one, even though I made sure not to start my server. But I suppose without a repro it's hard to tell what could've gone wrong.

[!NOTE] I just realized that the WebSocket issue is most likely due to the development value. My server file is creating Vite dev server middleware in that case, which is not intended for tests to run.

Reproduction

https://github.com/silvenon/repro/tree/vitest-extension-node-env

For WebSocket I'll file a separate issue.

System Info

System:
    OS: macOS 14.4
    CPU: (8) arm64 Apple M1 Pro
    Memory: 51.06 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.19.1 - ~/.n/bin/node
    npm: 10.2.4 - ~/.n/bin/npm
    pnpm: 8.6.10 - /opt/homebrew/bin/pnpm
    bun: 1.0.31 - ~/.bun/bin/bun
  IDEs:
    VSCode: 1.87.2 - /usr/local/bin/code
    Vim: 9.0 - /usr/bin/vim
    Xcode: /undefined - /usr/bin/xcodebuild
  Browsers:
    Chrome: 122.0.6261.129
    Edge: 122.0.2365.92
    Safari: 17.4
  npmPackages:
    @vitejs/plugin-react: ^4.2.0 => 4.2.0 
    @vitest/ui: ^1.2.2 => 1.4.0 
    vite: ^5.0.0 => 5.0.0 
    vitest: ^1.4.0 => 1.4.0

Used Package Manager

npm

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

sheremet-va commented 5 months ago

NODE_ENV is set to test by Vitest.

silvenon commented 5 months ago

Not while running tests via the extension, because my tests that rely on this are failing (because of that). Running the same tests via the Vitest CLI works.

silvenon commented 5 months ago

In fact, logging process.env.NODE_ENV in beforeAll ends up being development to me 😮

This is then also the source of my WebSocket issue because my tests are createing Vite dev server middleware in case of development.

silvenon commented 5 months ago

Here's the repro: https://github.com/silvenon/repro/tree/vitest-extension-node-env

hi-ogawa commented 5 months ago

It looks like NODE_ENV ??= 'test' happens in startVitest API which is a wrapper of createVitest API used by vscode extension:

https://github.com/vitest-dev/vitest/blob/5c7e9ca05491aeda225ce4616f06eefcd068c0b4/packages/vitest/src/node/cli/cli-api.ts#L31-L55

export async function startVitest(...): Promise<Vitest | undefined> {
  process.env.TEST = 'true'
  process.env.VITEST = 'true'
  process.env.NODE_ENV ??= 'test'

  ...

  const ctx = await createVitest(mode, options, viteOverrides, vitestOptions)

When NODE_ENV is empty, if I remember correctly, Vite injects "development" by default, so probably that's what's happening currently.

For the time being, I think using vitest.nodeEnv should work (at least for test code, but maybe not if it's globalSetup etc...).

// .vscode/settings.json
{
  "vitest.nodeEnv": {
    "NODE_ENV": "test"
  }
}

I'm not sure where the fix should belong to. Whether adding the same process.env.... routine to vscode extension worker or createVitest should have this routine instead of startVitest wrapper.

I'm thinking we can fix createVitest side on Vitest, but maybe we should do both since users might not upgrade Vitest.

vitest-dev / vscode