search

stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.55k stars 242 forks source link

`@stryker-mutator/mocha-runner` incompatible with Mocha v10.5.2 #4883

Open ericcornelissen opened 5 days ago

ericcornelissen commented 5 days ago

Summary

After upgrading from Mocha v10.2.0 to Mocha v10.5.2 the @stryker-mutator/mocha-runner started producing an error indicating incompatibility. In particular it seems to be calling testFileNames.forEach which no longer exists:

05:32:20 (2233) INFO ProjectReader Found 3 of 56 file(s) to be mutated using incremental report with 410 mutant(s), and 297 test(s).
05:32:20 (2233) INFO Instrumenter Instrumented 3 source file(s) with 410 mutant(s)
05:32:20 (2233) INFO ConcurrencyTokenProvider Creating 2 checker process(es) and 2 test runner process(es).
05:32:23 (2233) INFO BroadcastReporter Detected that current console does not support the "progress" reporter, downgrading to "progress-append-only" reporter
05:32:24 (2233) ERROR Stryker Unexpected error occurred while running Stryker StrykerError: TypeError: testFileNames.forEach is not a function
TypeError: testFileNames.forEach is not a function
    at MochaTestRunner.init (file:///home/runner/work/eslint-plugin-top/eslint-plugin-top/node_modules/@stryker-mutator/mocha-runner/src/mocha-test-runner.ts:78:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ChildProcessTestRunnerWorker.init (file:///home/runner/work/eslint-plugin-top/eslint-plugin-top/node_modules/@stryker-mutator/core/src/test-runner/child-process-test-runner-worker.ts:31:7)
    at async ChildProcessProxyWorker.handleCall (file:///home/runner/work/eslint-plugin-top/eslint-plugin-top/node_modules/@stryker-mutator/core/src/child-proxy/child-process-proxy-worker.ts:98:22)
    at ChildProcess.<anonymous> (file:///home/runner/work/eslint-plugin-top/eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/child-proxy/child-process-proxy.js:122:72)
    at ChildProcess.emit (node:events:520:28)
    at emit (node:internal/child_process:951:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21) {
  innerError: undefined
}
05:32:24 (2233) INFO Stryker This might be a known problem with a solution documented in our troubleshooting guide.
05:32:24 (2233) INFO Stryker You can find it at https://stryker-mutator.io/docs/stryker-js/troubleshooting/
05:32:24 (2233) INFO Stryker Still having trouble figuring out what went wrong? Try `npx stryker run --fileLogLevel trace --logLevel debug` to get some more info.

Stryker config

module.exports = {
  coverageAnalysis: 'perTest',
  inPlace: false,
  ignoreStatic: true,
  mutate: ['lib/**/*.ts', '!lib/index.ts', '!lib/configs/*.ts'],

  testRunner: 'mocha',
  mochaOptions: {
    config: '.mocharc.yml',
    spec: ['tests/unit/**/*.test.ts']
  },

  incremental: true,
  incrementalFile: '.cache/stryker-incremental.json',

  disableTypeChecks: '{lib,tests}/**/*.ts',
  checkers: ['typescript'],
  tsconfigFile: 'tsconfig.json',

  reporters: ['clear-text', 'html', 'progress'],
  htmlReporter: {
    fileName: '_reports/mutation/index.html'
  },
  thresholds: {
    high: 100,
    low: 100,
    break: 100
  },

  tempDirName: '.temp/stryker',
  cleanTempDir: true
};

Test runner config

extension:
  - ts
require: ts-node/register

Stryker environment

+-- @stryker-mutator/core@8.2.6
+-- @stryker-mutator/mocha-runner@8.2.6
+-- @stryker-mutator/typescript-checker@8.2.6
+-- mocha@10.5.2

Test runner environment

npm test

Your Environment

software version(s)
node 22.3.0
npm 10.8.1
Operating System Ubuntu 22.04

Add stryker.log

09:14:29 (78) DEBUG ConfigReader Loading config from stryker.config.js
09:14:29 (78) DEBUG ConfigReader Loaded config: {
  "coverageAnalysis": "perTest",
  "inPlace": false,
  "ignoreStatic": true,
  "mutate": [
    "lib/**/*.ts",
    "!lib/index.ts",
    "!lib/configs/*.ts"
  ],
  "testRunner": "mocha",
  "mochaOptions": {
    "config": ".mocharc.yml",
    "spec": [
      "tests/unit/**/*.test.ts"
    ]
  },
  "incremental": true,
  "incrementalFile": ".cache/stryker-incremental.json",
  "disableTypeChecks": "{lib,tests}/**/*.ts",
  "checkers": [
    "typescript"
  ],
  "tsconfigFile": "tsconfig.json",
  "reporters": [
    "clear-text",
    "html",
    "progress"
  ],
  "htmlReporter": {
    "fileName": "_reports/mutation/index.html"
  },
  "thresholds": {
    "high": 100,
    "low": 100,
    "break": 100
  },
  "tempDirName": ".temp/stryker",
  "cleanTempDir": true,
  "fileLogLevel": "trace",
  "configFile": "stryker.config.js",
  "allowConsoleColors": true,
  "checkerNodeArgs": [],
  "commandRunner": {
    "command": "npm test"
  },
  "clearTextReporter": {
    "allowColor": true,
    "allowEmojis": false,
    "logTests": true,
    "maxTestsToLog": 3,
    "reportTests": true,
    "reportMutants": true,
    "reportScoreTable": true
  },
  "dashboard": {
    "baseUrl": "https://dashboard.stryker-mutator.io/api/reports",
    "reportType": "full"
  },
  "dryRunOnly": false,
  "eventReporter": {
    "baseDir": "reports/mutation/events"
  },
  "ignorePatterns": [],
  "force": false,
  "logLevel": "info",
  "maxConcurrentTestRunners": 9007199254740991,
  "maxTestRunnerReuse": 0,
  "mutator": {
    "plugins": null,
    "excludedMutations": []
  },
  "plugins": [
    "@stryker-mutator/*"
  ],
  "appendPlugins": [],
  "jsonReporter": {
    "fileName": "reports/mutation/mutation.json"
  },
  "symlinkNodeModules": true,
  "testRunnerNodeArgs": [],
  "timeoutFactor": 1.5,
  "timeoutMS": 5000,
  "dryRunTimeoutMinutes": 5,
  "warnings": true,
  "disableBail": false,
  "allowEmpty": false,
  "ignorers": []
}
09:14:29 (78) DEBUG PluginLoader Loading @stryker-mutator/* from /eslint-plugin-top/node_modules/@stryker-mutator
09:14:29 (78) DEBUG PluginLoader Loading plugin "@stryker-mutator/mocha-runner" (matched with expression @stryker-mutator/*)
09:14:29 (78) DEBUG PluginLoader Loading plugin "@stryker-mutator/typescript-checker" (matched with expression @stryker-mutator/*)
09:14:29 (78) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:29 (78) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:29 (78) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:29 (78) DEBUG MetaSchemaBuilder Contributing 2 schemas from plugins to options validation.
09:14:29 (78) INFO ProjectReader Found 3 of 59 file(s) to be mutated using incremental report with 410 mutant(s), and 297 test(s).
09:14:29 (78) DEBUG ProjectReader All input files: [
  "/eslint-plugin-top/.c8rc.json",
  "/eslint-plugin-top/.cache/result.json",
  "/eslint-plugin-top/.editorconfig",
  "/eslint-plugin-top/.eslintrc.js",
  "/eslint-plugin-top/.gitattributes",
  "/eslint-plugin-top/.github/codeql.yml",
  "/eslint-plugin-top/.github/dependabot.yml",
  "/eslint-plugin-top/.github/labeler.yml",
  "/eslint-plugin-top/.github/workflows/audit.yml",
  "/eslint-plugin-top/.github/workflows/check.yml",
  "/eslint-plugin-top/.github/workflows/labeler.yml",
  "/eslint-plugin-top/.github/workflows/nightly.yml",
  "/eslint-plugin-top/.github/workflows/publish.yml",
  "/eslint-plugin-top/.github/workflows/release.yml",
  "/eslint-plugin-top/.github/workflows/reusable-audit.yml",
  "/eslint-plugin-top/.github/workflows/secrets.yml",
  "/eslint-plugin-top/.github/workflows/semgrep.yml",
  "/eslint-plugin-top/.github/workflows/weekly.yml",
  "/eslint-plugin-top/.gitignore",
  "/eslint-plugin-top/.licensee.json",
  "/eslint-plugin-top/.lockfile-lintrc.yml",
  "/eslint-plugin-top/.markdownlint.yml",
  "/eslint-plugin-top/.mocharc.yml",
  "/eslint-plugin-top/.npmignore",
  "/eslint-plugin-top/.npmrc",
  "/eslint-plugin-top/.nsprc",
  "/eslint-plugin-top/.nvmrc",
  "/eslint-plugin-top/.prettierrc.yml",
  "/eslint-plugin-top/.tool-versions",
  "/eslint-plugin-top/.vscode/settings.json",
  "/eslint-plugin-top/CHANGELOG.md",
  "/eslint-plugin-top/CONTRIBUTING.md",
  "/eslint-plugin-top/LICENSE",
  "/eslint-plugin-top/README.md",
  "/eslint-plugin-top/RELEASE.md",
  "/eslint-plugin-top/SECURITY.md",
  "/eslint-plugin-top/docs/rules/no-top-level-side-effects.md",
  "/eslint-plugin-top/docs/rules/no-top-level-variables.md",
  "/eslint-plugin-top/index.js",
  "/eslint-plugin-top/knip.jsonc",
  "/eslint-plugin-top/lib/configs/recommended.ts",
  "/eslint-plugin-top/lib/configs/strict.ts",
  "/eslint-plugin-top/lib/helpers.ts",
  "/eslint-plugin-top/lib/index.ts",
  "/eslint-plugin-top/lib/rules/no-top-level-side-effects.ts",
  "/eslint-plugin-top/lib/rules/no-top-level-variables.ts",
  "/eslint-plugin-top/package-lock.json",
  "/eslint-plugin-top/package.json",
  "/eslint-plugin-top/rollup.config.js",
  "/eslint-plugin-top/scripts/bump-changelog.js",
  "/eslint-plugin-top/scripts/maybe-run.js",
  "/eslint-plugin-top/stryker.config.js",
  "/eslint-plugin-top/tests/compat/helpers.ts",
  "/eslint-plugin-top/tests/compat/main.test.ts",
  "/eslint-plugin-top/tests/compat/snapshots.ts",
  "/eslint-plugin-top/tests/unit/helpers.ts",
  "/eslint-plugin-top/tests/unit/no-top-level-side-effects.test.ts",
  "/eslint-plugin-top/tests/unit/no-top-level-variables.test.ts",
  "/eslint-plugin-top/tsconfig.json"
]
09:14:29 (78) DEBUG ProjectReader Files to mutate: [
  "/eslint-plugin-top/lib/helpers.ts",
  "/eslint-plugin-top/lib/rules/no-top-level-side-effects.ts",
  "/eslint-plugin-top/lib/rules/no-top-level-variables.ts"
]
09:14:29 (78) DEBUG TemporaryDirectory Using temp directory "/eslint-plugin-top/.temp/stryker"
09:14:29 (78) DEBUG Instrumenter Instrumenting 3 source files with mutants
09:14:29 (78) DEBUG Instrumenter Instrumented lib/helpers.ts (9 mutant(s))
09:14:29 (78) DEBUG Instrumenter Instrumented lib/rules/no-top-level-side-effects.ts (288 mutant(s))
09:14:29 (78) DEBUG Instrumenter Instrumented lib/rules/no-top-level-variables.ts (113 mutant(s))
09:14:29 (78) INFO Instrumenter Instrumented 3 source file(s) with 410 mutant(s)
09:14:30 (78) DEBUG TSConfigPreprocessor Rewriting file ProjectFile {
  fs: [FileSystem],
  name: '/eslint-plugin-top/tsconfig.json',
  mutate: false
}
09:14:30 (78) INFO ConcurrencyTokenProvider Creating 8 checker process(es) and 7 test runner process(es).
09:14:30 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 0 with pid 89 
09:14:30 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 1 with pid 90 
09:14:30 (89) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:30 (89) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:30 (89) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:30 (90) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:30 (90) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:30 (90) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:32 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 2 with pid 111 
09:14:32 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 3 with pid 122 
09:14:32 (111) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:32 (111) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:32 (111) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:32 (122) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:32 (122) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:32 (122) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:34 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 4 with pid 133 
09:14:34 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 5 with pid 144 
09:14:34 (133) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:34 (133) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:34 (133) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:34 (144) DEBUG PluginLoader Loading plugin @stryker-mutator/mocha-runner
09:14:34 (144) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
09:14:34 (144) DEBUG PluginLoader Loading plugin file:///eslint-plugin-top/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
09:14:36 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 6 with pid 155 
09:14:36 (78) DEBUG ChildProcessProxy Started CheckerWorker in worker process 7 with pid 162
regseb commented 5 days ago

I think it was the pull request mochajs/mocha#5086 (released in Mocha v10.5.1) that broke Stryker. The collect-files method no longer returns the list of files, but an object with the list of files and the list of unmatchedFiles:

-   return files;
+   return {
+     files,
+     unmatchedFiles
+   };