PNPM can't resolve coverage-v8 or coverage-istanbul in CI environment

pgrones commented 3 months ago

Describe the bug

I'm not sure whether this is an issue with vitest or with pnpm.

When running tests with coverage locally, everything seems to work fine. When running the same command in a CI environment however, the execution fails, printing the following error message.

What's strange is that this doesn't seem to happen for every project. One of our other projects has virtually the same setup and config files and runs without any issues.

> vitest --watch=false --reporter=junit --coverage
⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯
Error: Cannot find package 'PathToApp\node_modules\.pnpm\@vitest+coverage-istanbul@1.5.2_vitest@1.5.2_@types+node@20.14.14_@vitest+ui@1.5.2_jsdom@24.1_vcfzxg5732ovub2uuh5swlnnaa\node_modules\istanbul-lib-coverage\package.json' imported from PathToApp\node_modules\.pnpm\@vitest+coverage-istanbul@1.5.2_vitest@1.5.2_@types+node@20.14.14_@vitest+ui@1.5.2_jsdom@24.1_vcfzxg5732ovub2uuh5swlnnaa\node_modules\@vitest\coverage-istanbul\dist\provider.js
Did you mean to import istanbul-lib-coverage@3.2.2/node_modules/istanbul-lib-coverage/index.js?
 ❯ legacyMainResolve ../node:internal/modules/esm/resolve:214:26
 ❯ packageResolve ../node:internal/modules/esm/resolve:840:14
 ❯ moduleResolve ../node:internal/modules/esm/resolve:910:20
 ❯ defaultResolve ../node:internal/modules/esm/resolve:1130:11
 ❯ ModuleLoader.defaultResolve ../node:internal/modules/esm/loader:[39]6:12
 ❯ ModuleLoader.resolve ../node:internal/modules/esm/loader:365:25
 ❯ ModuleLoader.getModuleJob ../node:internal/modules/esm/loader:2[40]:38
 ❯ ModuleWrap.<anonymous> ../node:internal/modules/esm/module_job:85:39
 ❯ link ../node:internal/modules/esm/module_job:84:36
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'ERR_MODULE_NOT_FOUND' }
 ELIFECYCLE  Command failed with exit code 1.

For completeness, here are all the other files that might produce the issue:

package.json

{
  "name": "nameofproject",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build:test": "tsc --noEmit && pnpm lint && vite build --mode test --sourcemap true",
    "build:production": "tsc --noEmit && pnpm lint && vite build",
    "test:watch": "vitest",
    "test:ui": "vitest --coverage --ui",
    "test:report": "vitest --watch=false --reporter=junit --coverage",
    "lint": "eslint ./src ./__tests__ --ext .jsx,.js,.ts,.tsx",
    "type-check": "tsc",
    "analyze:bundle": "source-map-explorer build/assets/**/*.js --no-border-checks"
  },
  "dependencies": {
    "@dbsd-shared/hypo-react-core": "^3.1.0",
    "@mantine/core": "^7.11.2",
    "@mantine/dates": "^7.11.2",
    "@mantine/form": "^7.11.2",
    "@mantine/hooks": "^7.11.2",
    "@mantine/modals": "^7.11.2",
    "@mantine/notifications": "^7.11.2",
    "@tanstack/react-query": "^5.51.21",
    "axios": "^1.7.3",
    "dayjs": "^1.11.12",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-icons": "^5.2.1",
    "react-router-dom": "^6.26.0"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^6.4.8",
    "@testing-library/react": "^15.0.5",
    "@testing-library/react-hooks": "^8.0.1",
    "@testing-library/user-event": "^14.5.2",
    "@types/node": "^20.14.14",
    "@types/react": "^18.3.3",
    "@types/react-dom": "^18.3.0",
    "@types/react-router-dom": "^5.3.3",
    "@typescript-eslint/eslint-plugin": "^7.18.0",
    "@typescript-eslint/parser": "^7.18.0",
    "@vitejs/plugin-react": "^4.3.1",
    "@vitest/coverage-istanbul": "1.5.2",
    "@vitest/ui": "^1.5.2",
    "eslint": "^8.57.0",
    "eslint-config-prettier": "^9.1.0",
    "eslint-plugin-import": "^2.29.1",
    "eslint-plugin-jsx-a11y": "^6.9.0",
    "eslint-plugin-react": "^7.35.0",
    "jsdom": "^24.1.1",
    "msw": "^2.3.5",
    "postcss": "^8.4.40",
    "postcss-preset-mantine": "^1.17.0",
    "postcss-simple-vars": "^7.0.1",
    "prettier": "^3.3.3",
    "react-test-renderer": "^18.3.1",
    "source-map-explorer": "^2.5.3",
    "typescript": "^5.5.4",
    "vite": "^5.3.5",
    "vite-tsconfig-paths": "^4.3.2",
    "vitest": "^1.5.2"
  }
}

vite.config.ts

/// <reference types="vitest" />
import react from '@vitejs/plugin-react';
import { defineConfig, loadEnv } from 'vite';
import tsconfigPaths from 'vite-tsconfig-paths';
import { configDefaults } from 'vitest/config';

export default defineConfig(({ mode }) => {
  const env = loadEnv(mode, './');

  const transformIndexHtml = () => {
    return {
      name: 'index-html-transform',
      transformIndexHtml(html: string) {
        return html.replace(/%(.*?)%/g, (_, p1) => env[p1]);
      }
    };
  };

  return {
    server: {
      host: true,
      port: 1234,
      strictPort: true
    },
    build: {
      emptyOutDir: true,
      outDir: './build'
    },
    test: {
      globals: true,
      root: 'src',
      environment: 'jsdom',
      setupFiles: '../__tests__/setup.ts',
      outputFile: '../test_result/junit.xml',
      reporters: ['default', 'junit'],
      silent: true,
      clearMocks: true,
      exclude: [...configDefaults.exclude, 'test/**'],
      coverage: {
        provider: 'istanbul',
        reporter: ['text-summary', 'html', 'cobertura'],
        reportsDirectory: '../test_coverage'
      }
    },
    plugins: [transformIndexHtml(), tsconfigPaths(), react()]
  };
});

tsconfig.json

{
  "compilerOptions": {
    "target": "es2022",
    "lib": ["dom", "dom.iterable", "esnext"],
    "allowJs": true,
    "skipLibCheck": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "strict": true,
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react-jsx",
    "noFallthroughCasesInSwitch": true,
    "downlevelIteration": true,
    "useUnknownInCatchVariables": false,
    "noUnusedParameters": true,
    "noUnusedLocals": true,
    "types": ["vitest/globals", "@testing-library/jest-dom"]
  },
  "include": ["src", "__tests__"]
}

Reproduction

Not possible without a CI environment

System Info

System:
    OS: Windows 10 10.0.20348
    CPU: (4) x64 Intel(R) Xeon(R) Gold 6242R CPU @ 3.10GHz
    Memory: 5.96 GB / 12.00 GB
  Binaries:
    Node: 20.11.1 - C:\Program Files\nodejs\node.EXE
    npm: 10.2.4 - C:\Program Files\nodejs\npm.CMD
    pnpm: 9.6.0 - C:\Program Files\nodejs\pnpm.CMD
  Browsers:
    Internet Explorer: 11.0.20348.2520

Used Package Manager

pnpm

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

github-actions[bot] commented 3 months ago

Hello @pgrones. Please provide a minimal reproduction using a GitHub repository or StackBlitz (you can also use examples). Issues marked with needs reproduction will be closed if they have no activity within 3 days.

AriPerkkio commented 3 months ago

Without being able to reproduce this issue there is nothing we can do. Everything looks fine in your linked configuration files.

Though I would recommend to make sure your project has pnpm-lock.yaml committed to git, and that CI is using it correctly.

pgrones commented 3 months ago

I figured it was a long shot without being reproducable, but we were starting to run out of ideas.

Looking at the lock file might be a good hint. Any recommendations on what flags to include? We always commit the lock file and are using the following command:

pnpm i --prefer-offline --frozen-lockfile

If you don't think this is an issue with vitest anyways, then you can close the issue. Thank you for taking the time to answer!

AriPerkkio commented 3 months ago

The error message here is quite cryptic but it might be good starting point to check that lock file doesn't have multiple different versions of Vitest packages used. Using --frozen-lockfile and checking that CI install everything without any warnings is good point too.

Keito654 commented 3 months ago

I'm having this problem too. One thing I've found is that downgrading pnpm to version 8 solves it.

pgrones commented 3 months ago

That doesn't seem to work for us. When downgrading to pnpm 8.15.9, the command just fails silently. Even using pnpm 8 locally and committing the lock file doesn't seem to help.

When connecting directly to the pipeline runner and executing vitest watch, the error now complains about not finding react-router-dom.

Something seems to be seriously wrong with pnpm for this project. The build with vite works fine though, so it's some weird combination of pnpm and vitest that causes the issue

pgrones commented 2 months ago

Figured it out. We're using Windows and the path pnpm generates is longer than the limit imposed by Windows.

The command only works locally, because the project isn't as deeply nested as it is on the runner.

Took a few hours of testing until we finally figured this out, but I'm glad it's not actually an issue with pnpm or vitest.

vitest-dev / vitest