vitest-dev / vscode

VS Code extension for Vitest
https://vitest.dev/vscode
MIT License
736 stars 83 forks source link

Running with coverage produces error #339

Closed ffMathy closed 3 months ago

ffMathy commented 4 months ago

Describe the bug

TypeError: Cannot read properties of null (reading 'result')
    at mergeProcessCovs (/workspaces/redacted/node_modules/.pnpm/@bcoe+v8-coverage@0.2.3/node_modules/@bcoe/v8-coverage/dist/lib/merge.js:22:44)
    at file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.4.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:266:22
    at async Promise.all (index 1)
    at async V8CoverageProvider.reportCoverage (file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.4.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:263:11)
    at async Vitest.reportCoverage (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:12014:7)
    at async file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11789:7
    at async Vitest.runFiles (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11795:12)
    at async t.rerunTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4352)
    at async t.runTestFiles (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4153)
    at async t.runTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:3622)

Reproduction

No repro yet - hoping the stack trace is enough.

Output

[INFO 9:47:27 AM] [v0.6.0] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[INFO 9:47:27 AM] [API] Running Vitest: v1.5.0 (/workspaces/WATS/vitest.workspace.ts)
[INFO 9:47:28 AM] [API] Vitest process 22069 created
[Worker]  Vitest  "cache.dir" is deprecated, use Vite's "cacheDir" instead if you want to change the cache director. Note caches will be written to "cacheDir/vitest"
[Worker] Coverage reports directory is generated: /tmp/vitest-coverage-240cb5ae-b6a8-477a-b9ea-0653864416cf
[INFO 9:47:36 AM] Running all tests in WATS

[INFO 9:48:54 AM] [TEST] run-test: redacted.test.ts  redacted 10.31s

[Error 9:49:23 AM] [Error TypeError] TypeError: Cannot read properties of null (reading 'result')
    at mergeProcessCovs (/workspaces/redacted/node_modules/.pnpm/@bcoe+v8-coverage@0.2.3/node_modules/@bcoe/v8-coverage/dist/lib/merge.js:22:44)
    at file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.4.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:266:22
    at async Promise.all (index 1)
    at async V8CoverageProvider.reportCoverage (file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.4.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:263:11)
    at async Vitest.reportCoverage (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:12014:7)
    at async file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11789:7
    at async Vitest.runFiles (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11795:12)
    at async t.rerunTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4352)
    at async t.runTestFiles (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4153)
    at async t.runTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:3622)

Version

v0.6.0

Validations

AriPerkkio commented 4 months ago
at file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.4.0_vitest@1.5.0
                                                                      ^^^^^        ^^^^^

Could you try if updating the coverage package works? Vitest expects that all Vitest packages have matching versions.

ffMathy commented 4 months ago

I get the same after upgrading:

[Error 3:49:10 PM] [Error TypeError] Cannot read properties of null (reading 'result')
TypeError: Cannot read properties of null (reading 'result')
    at mergeProcessCovs (/workspaces/redacted/node_modules/.pnpm/@bcoe+v8-coverage@0.2.3/node_modules/@bcoe/v8-coverage/dist/lib/merge.js:22:44)
    at file:///workspaces/redacted/node_modules/.pnpm/@vitest+coverage-v8@1.5.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:2374:22
    at async Promise.all (index 0)
    at async V8CoverageProvider.reportCoverage (file:///workspaces/WATS/node_modules/.pnpm/@vitest+coverage-v8@1.5.0_vitest@1.5.0/node_modules/@vitest/coverage-v8/dist/provider.js:2371:11)
    at async Vitest.reportCoverage (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:12014:7)
    at async file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11789:7
    at async Vitest.runFiles (file:///workspaces/redacted/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.6_@vitest+ui@1.4.0_jsdom@24.0.0_less@4.1.3_stylus@0.59.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11795:12)
    at async t.rerunTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4352)
    at async t.runTestFiles (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:4153)
    at async t.runTests (/root/.vscode-server/extensions/vitest.explorer-0.6.0/dist/worker.js:3:3622)
ffMathy commented 4 months ago

Here's my vitest.workspace.ts file:

import { nxViteTsPaths } from "@nx/vite/plugins/nx-tsconfig-paths.plugin";
import { v4 } from 'uuid';
import { UserConfig, defineWorkspace } from "vitest/config";

const userConfig: UserConfig = {
  plugins: [nxViteTsPaths()],
  test: {
    env: {
      VITEST: v4()
    },
    pool: 'forks',
    testTimeout: 60_000 * 3,
    maxConcurrency: 3,
    maxWorkers: 10,
    hookTimeout: 60_000 * 3,
    globals: true,
    globalSetup: ['./jest.global-setup.ts'],
    setupFiles: ['./jest.setup.ts'],
    cache: {
      dir: '../../../node_modules/.vitest',
    },
    environment: 'node',
    include: ['src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
    exclude: ['src/apps/widgets/**/*']
  },
  esbuild: {
    target: 'es2022'
  },
};

export default defineWorkspace([
  userConfig
]);
ludovicmnji commented 4 months ago

I have the same issue :

[Error 21:15:15] [Error TypeError] Cannot read properties of null (reading 'result')
TypeError: Cannot read properties of null (reading 'result')
    at mergeProcessCovs (/Users/ludovic/Projets/starter/node_modules/.pnpm/@bcoe+v8-coverage@0.2.3/node_modules/@bcoe/v8-coverage/dist/lib/merge.js:22:44)
    at file:///Users/ludovic/Projets/starter/node_modules/.pnpm/@vitest+coverage-v8@1.5.0_vitest@1.5.0_@types+node@20.12.7_jsdom@24.0.0_/node_modules/@vitest/coverage-v8/dist/provider.js:2374:22
    at async Promise.all (index 0)
    at async V8CoverageProvider.reportCoverage (file:///Users/ludovic/Projets/starter/node_modules/.pnpm/@vitest+coverage-v8@1.5.0_vitest@1.5.0_@types+node@20.12.7_jsdom@24.0.0_/node_modules/@vitest/coverage-v8/dist/provider.js:2371:11)
    at async Vitest.reportCoverage (file:///Users/ludovic/Projets/starter/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.7_jsdom@24.0.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:12014:7)
    at async file:///Users/ludovic/Projets/starter/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.7_jsdom@24.0.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11789:7
    at async Vitest.runFiles (file:///Users/ludovic/Projets/starter/node_modules/.pnpm/vitest@1.5.0_@types+node@20.12.7_jsdom@24.0.0/node_modules/vitest/dist/vendor/cac.qnW6GNL2.js:11795:12)
    at async t.rerunTests (/Users/ludovic/.vscode/extensions/vitest.explorer-0.6.2/dist/worker.js:3:4427)
    at async t.runTestFiles (/Users/ludovic/.vscode/extensions/vitest.explorer-0.6.2/dist/worker.js:3:4228)
    at async t.runTests (/Users/ludovic/.vscode/extensions/vitest.explorer-0.6.2/dist/worker.js:3:3697)
[Worker] /Users/ludovic/Projets/starter/node_modules/.pnpm/@bcoe+v8-coverage@0.2.3/node_modules/@bcoe/v8-coverage/dist/lib/merge.js:22
        for (const scriptCov of processCov.result) {
                                           ^

If it can help to find the issue cause, this is my configuration :

// vitest.workspace.ts
import { defineWorkspace } from "vitest/config";

export default defineWorkspace(["apps/*", "packages/*"]);
// vitest.config.ts
import { coverageConfigDefaults, defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    coverage: {
      provider: "v8",
      exclude: [
        ...coverageConfigDefaults.exclude,
        "commitlint.config.ts",
        "**/vitest-setup.ts",
        "tooling/*",
        "packages/utils/src/index.ts",
        "**/*.stories.tsx",
      ],
    },
  },
});

I use @vitest/coverage-v8 and vitest v1.5.0

alvis commented 4 months ago

me the same. In the log, I can see a temporary folder for the coverage has been created. However, after running a coverage test, I only see 2 coverage files (e.g. /var/folders/s6/3j1v_rn101x9f5m7w63bssb80000gn/T/vitest-coverage-9bf5988c-bdc1-436f-a101-d9a8d7337d36/.tmp/coverage-0.json) under .tmp folder of the temporary coverage folder, and the content of both of the files are simply null.

When I run vitest run --coverage, the coverage files are generated as expected. @sheremet-va Do you know how to print out the command used by the vscode extension to generate the coverage result to the log?

sheremet-va commented 4 months ago

Do you know how to print out the command used by the vscode extension to generate the coverage result to the log?

It uses programmatic API with your config but overrides the directory.

Is there any good reproduction I can check to debug this issue?

github-actions[bot] commented 4 months ago

Hello @ffMathy. Please provide a minimal reproduction using a GitHub repository or StackBlitz (you can also use examples). Issues marked with needs reproduction will be closed if they have no activity within 3 days.

ffMathy commented 4 months ago

Do you know how to print out the command used by the vscode extension to generate the coverage result to the log?

It uses programmatic API with your config but overrides the directory.

Is there any good reproduction I can check to debug this issue?

No, unfortunately not. But as you can see, others are also having the issue, so I think maybe closing this in 3 days seems a bit rushed?

sheremet-va commented 4 months ago

I cannot start fixing the issue if there is no code to try.

ffMathy commented 4 months ago

No, but at least we can then speculate and discuss theories around what the root cause could be, and dive into it together.

Is there any way where we can insert some kind of logging to help determine the cause of this?

sheremet-va commented 4 months ago

No, but at least we can then speculate and discuss theories around what the root cause could be, and dive into it together.

You can still speculate, the issue is not locked. More info on why I require that every issue has a reproduction: https://antfu.me/posts/why-reproductions-are-required

Is there any way where we can insert some kind of logging to help determine the cause of this?

You can add DEBUG: "vite:*,vitest:coverage" to vitest.nodeEnv option.

alvis commented 4 months ago

@ffMathy @ludovicmnji By any chance you have multiple configs in your monorepo? When I remove all configs but only one left on the workspace root, it works perfectly

ffMathy commented 4 months ago

No, but at least we can then speculate and discuss theories around what the root cause could be, and dive into it together.

You can still speculate, the issue is not locked. More info on why I require that every issue has a reproduction: https://antfu.me/posts/why-reproductions-are-required

I get that. I'm an open source maintainer myself (with 100+ NuGet packages and 10+ NPM packages under my disposal). It can be tiring and people can be very mean.

However, I am just saying that you probably also have an interest in getting good ratings for the extension, right? It's not all black and white. I get where you are coming from, but this issue now has 3 upvotes (despite it being closed), so it might be affecting a lot of people.

@alvis I'll try that soon and report back.

ffMathy commented 4 months ago

@alvis that unfortunately didn't work for me. I just have one Vitest workspace file in the root of my repo.

alvis commented 4 months ago

@ffMathy do you have a config file alongside your workspace file?

here is my config

// vitest.workspace.ts
import { defineWorkspace } from "vitest/config";

export default defineWorkspace(["**/vitest.config.ts"]);

// vitest.config.ts
import tsconfigPaths from 'vite-tsconfig-paths';
import { defineConfig } from 'vitest/config';

export default defineConfig({
  plugins: [tsconfigPaths()],
  test: {
    watch: false,
    coverage: {
      all: true,
      clean: true,
      include: ['**/source/**'],
      provider: 'v8',
      reporter: ['lcov', 'text', 'html'],
    },
    include: ['**/*.(spec|spec-d).?(c|m)[jt]s?(x)'], 
  },
});
ffMathy commented 4 months ago

No, I only have a workspace file (the one I posted in an earlier comment) ❤️🙏

ludovicmnji commented 4 months ago

@ffMathy @ludovicmnji By any chance you have multiple configs in your monorepo? When I remove all configs but only one left on the workspace root, it works perfectly

I have multiple configs (monorepo config). I have vitest.workspace.ts + vitest.config.ts in root (because coverage in monorepo can't be configured in each package) and I have vitest.config.ts in multiple packages.

motss commented 4 months ago

I'm also facing this issue with a monorepo and never realized that the vitest vscode extension already provides basic coverage support. I'm still trying to figure the issue out so do not expect a repro anytime soon.

My finding on the TypeError: Cannot read properties of null (reading 'result') error message was because of coverage files being stored with a null string and when this LoC in coverage-v8 package is trying to read the coverage file what it got was just a null object and mergeProcessCovs expect an object { result: [] }.

173   const contents = await fs.readFile(filename, 'utf-8'); // the file contains `null`
174   const coverage = JSON.parse(contents) as RawCoverage; // coverage = null
175   merged = mergeProcessCovs([merged, coverage]) // coverage should be at least `{ result: [] }`

Even though I am not 100% sure why the coverage file contains null and I'm still trying to debug. IMHO L174-L175 should have a basic check to ensure that coverage is not nullish before it tries to merge the coverage.

sheremet-va commented 4 months ago

IMHO L174-L175 should have a basic check to ensure that coverage is not nullish before it tries to merge the coverage.

It should not do that. The coverage value should never be null, there is a check for that when it's stored. If it is null at this point, then something went wrong prior to that. If we have a check here, the issue will just be swallowed and harder to debug.

motss commented 4 months ago

It should not do that. The coverage value should never be null, there is a check for that when it's stored. If it is null at this point, then something went wrong prior to that. If we have a check here, the issue will just be swallowed and harder to debug.

Good point. IIUC, the coverage here is where the coverage file was first created after each test. coverage here is returning null which explains why we got null from the coverage file.

Good news is that I was able to reproduce the error, which was quite unexpected. Based from the provided config files above I can deduce that the root cause of the coverage content being nullish was because of test.coverage.enabled was not set to true in vite.config.ts. I can consistently reproduce the error message and while you might not need a repro for that, you can use this repo for verification if needed.

https://github.com/vitest-dev/vscode/assets/10607759/1cdc3e14-3999-40ca-8571-125c49477089

I’m not sure how to handle this. An easy approach might be to provide some hints or simply enable coverage, regardless of the value of test.coverage.enabled in vite.config.ts.

Another possible root cause could be incorrect configuration of vite.workspace.ts. The workspace config confuses me and I question the need of it.

ludovicmnji commented 3 months ago

@motss I updated my vitest.config.ts file with test.coverage.enabled in the monorepo root, and it's working now, too.

// vitest.config.ts
import { coverageConfigDefaults, defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    coverage: {
      enabled: true,
      provider: "v8",
      exclude: [
        ...coverageConfigDefaults.exclude,
        "commitlint.config.ts",
        "**/vitest-setup.ts",
        "tooling/*",
        "packages/utils/src/index.ts",
        "**/*.stories.tsx",
      ],
    },
  },
});

But I don't want to run coverage when I simply run pnpm run test.