search

stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.6k stars 251 forks source link

`coverageAnalysis` with default value of `perTest` errors with `ReferenceError: document is not defined` #3982

Open mcmxcdev opened 1 year ago

mcmxcdev commented 1 year ago

Summary

As described in the issue title, the mutation run errors out and crashes with a stack trace pointing to @testing-library/react

Changing coverageAnalysis to off gets rid of the error and the mutation testing works as expected.

Stryker config

const config = {
  packageManager: 'pnpm',
  testRunner: 'jest',
  testRunnerNodeArgs: ['--experimental-vm-modules'],
  // concurrency: 3,
  coverageAnalysis: 'perTest',
  incremental: true,
  plugins: [
    '@stryker-mutator/jest-runner',
    '@stryker-mutator/typescript-checker',
  ],
  mutate: [
    'apps/project/components/plans/**/*.ts?(x)',
    '!apps/project/components/plans/**/*.spec.ts?(x)',
  ],
  jest: {
    configFile: 'apps/project/jest.config.ts',
  },
  mutator: {
    excludedMutations: [
      'UnaryOperator',
      'ObjectLiteral',
      'ArrowFunction',
      'StringLiteral',
    ],
  },
  checkers: ['typescript'],
  tsconfigFile: 'tsconfig.base.json',
};
export default config;

Test runner config

Stryker environment

    "@stryker-mutator/api": "^6.3.1",
    "@stryker-mutator/core": "^6.3.1",
    "@stryker-mutator/jest-runner": "^6.3.1",
    "@stryker-mutator/typescript-checker": "^6.3.1",

Test runner environment

jest

Your Environment

software version(s)
node v18.12.1
npm 8.19.2
Operating System Ubuntu 22.10

Add stryker.log

16:45:45 (461656) INFO ProjectReader No incremental result file found at reports/stryker-incremental.json, a full mutation testing run will be performed.
16:45:45 (461656) INFO ProjectReader Found 4 of 2073 file(s) to be mutated.
16:45:45 (461656) INFO Instrumenter Instrumented 4 source file(s) with 130 mutant(s)
16:45:45 (461656) INFO ConcurrencyTokenProvider Creating 8 checker process(es) and 7 test runner process(es).
16:45:49 (461656) INFO DryRunExecutor Starting initial test run (jest test runner with "perTest" coverage analysis). This may take a while.
16:45:55 (461656) ERROR DryRunExecutor One or more tests failed in the initial test run:
    PlansFilters should render successfully
        ReferenceError: document is not defined
    at render (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/@testing-library+react@14.0.0_react-dom@18.2.0_react@18.2.0/node_modules/@testing-library/react/dist/pure.js:215:5)
    at Object.<anonymous> (/home/user/Webdevelopment/Freelancing/org/repo/.stryker-tmp/sandbox726489/apps/repo-ui/components/plans/PlansFilters.spec.tsx:64:11)
    at Promise.then.completed (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:290:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:223:10)
    at _callCircusTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:248:40)
    at async _runTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:184:3)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:86:9)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:81:9)
    at async run (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:26:3)
    PlansFiltersItemRadioRadio should render successfully
        ReferenceError: document is not defined
    at render (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/@testing-library+react@14.0.0_react-dom@18.2.0_react@18.2.0/node_modules/@testing-library/react/dist/pure.js:215:5)
    at Object.<anonymous> (/home/user/Webdevelopment/Freelancing/org/repo/.stryker-tmp/sandbox726489/apps/repo-ui/components/plans/PlansFiltersItemRadio.spec.tsx:12:11)
    at Promise.then.completed (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:290:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:223:10)
    at _callCircusTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:248:40)
    at async _runTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:184:3)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:86:9)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:81:9)
    at async run (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:26:3)
    PlansFiltersItem should render successfully
        ReferenceError: document is not defined
    at render (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/@testing-library+react@14.0.0_react-dom@18.2.0_react@18.2.0/node_modules/@testing-library/react/dist/pure.js:215:5)
    at Object.<anonymous> (/home/user/Webdevelopment/Freelancing/org/repo/.stryker-tmp/sandbox726489/apps/repo-ui/components/plans/PlansFiltersItem.spec.tsx:12:11)
    at Promise.then.completed (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:290:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/utils.js:223:10)
    at _callCircusTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:248:40)
    at async _runTest (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:184:3)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:86:9)
    at async _runTestsForDescribeBlock (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:81:9)
    at async run (/home/user/Webdevelopment/Freelancing/org/repo/node_modules/.pnpm/jest-circus@29.4.1/node_modules/jest-circus/build/run.js:26:3)
16:45:55 (461656) ERROR Stryker There were failed tests in the initial test run.
de-don commented 1 year ago

In docs, there is a warning, that coverage analysis is not supported for jest, maybe this is a source of problems?

https://stryker-mutator.io/blog/typescript-coverage-analysis-support/#coverage-analysis-all

image