search

stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.59k stars 250 forks source link

Unable to run the jest configuration with Stryker on a custom Typescript project #5049

Open Fulcinator opened 1 week ago

Fulcinator commented 1 week ago

Hello, I am trying to run mutation testing on a custom project with Stryker. When running Stryker no tests are found, even though the test configuration with jest on its own is working.

The excerpt of the trace that I have is this:

13:13:43 (41680) DEBUG jestTestAdapterFactory Detected Jest version 29.7.0
13:13:43 (41680) DEBUG JestTestRunner Running jest with --findRelatedTests flag. Set jest.enableFindRelatedTests to false to run all tests on every mutant.
13:13:43 (41680) DEBUG CustomJestConfigLoader Read config from "jest.config.cjs" (used native `readInitialOptions` from jest-config).
13:13:43 (41680) DEBUG JestTestRunner Merging file-based config {
  "verbose": false,
  "testMatch": [
    "<rootDir>/src/**/test.ts"
  ],
  "moduleFileExtensions": [
    "js",
    "mjs",
    "ts"
  ],
  "testPathIgnorePatterns": [
    "<rootDir>/index.index.index.index.index.index.index.index.index.index.index.index.index.index.test.js"
  ],
  "transform": {
    "^.+\\.tsx?$": "ts-jest",
    "^.+\\.jsx?$": "babel-jest",
    "^.+\\.mjsx?$": "babel-jest",
    "^.+\\.mjs$": "babel-jest",
    "^.+\\.js$": "babel-jest"
  },
  "rootDir": "C:\\Users\\d063684\\Desktop\\Esperimento\\Project\\date-experiment-2024-mutation\\stryker-temp\\sandbox8668689"
}
      with custom config {}
      and default (internal) stryker config {
  "collectCoverage": false,
  "verbose": false,
  "notify": false,
  "bail": false,
  "reporters": []
}
13:13:43 (33628) DEBUG PluginLoader Loading plugin @stryker-mutator/jest-runner
13:13:43 (33628) DEBUG PluginLoader Loading plugin @stryker-mutator/typescript-checker
13:13:43 (33628) DEBUG PluginLoader Loading plugin file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/instrumenter/dist/src/frameworks/index.js
13:13:43 (33628) DEBUG PluginLoader Loading plugin file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/core/dist/src/reporters/index.js
13:13:44 (33628) DEBUG ChildProcessProxyWorker Changing current working directory for this process to C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\stryker-temp\sandbox8668689
13:13:44 (33628) DEBUG jestTestAdapterFactory Detected Jest version 29.7.0
13:13:44 (33628) DEBUG JestTestRunner Running jest with --findRelatedTests flag. Set jest.enableFindRelatedTests to false to run all tests on every mutant.
13:13:44 (33628) DEBUG CustomJestConfigLoader Read config from "jest.config.cjs" (used native `readInitialOptions` from jest-config).
13:13:44 (33628) DEBUG JestTestRunner Merging file-based config {
  "verbose": false,
  "testMatch": [
    "<rootDir>/src/**/test.ts"
  ],
  "moduleFileExtensions": [
    "js",
    "mjs",
    "ts"
  ],
  "testPathIgnorePatterns": [
    "<rootDir>/index.index.index.index.index.index.index.index.index.index.index.index.index.index.test.js"
  ],
  "transform": {
    "^.+\\.tsx?$": "ts-jest",
    "^.+\\.jsx?$": "babel-jest",
    "^.+\\.mjsx?$": "babel-jest",
    "^.+\\.mjs$": "babel-jest",
    "^.+\\.js$": "babel-jest"
  },
  "rootDir": "C:\\Users\\d063684\\Desktop\\Esperimento\\Project\\date-experiment-2024-mutation\\stryker-temp\\sandbox8668689"
}
      with custom config {}
      and default (internal) stryker config {
  "collectCoverage": false,
  "verbose": false,
  "notify": false,
  "bail": false,
  "reporters": []
}
13:13:47 (1428) TRACE ChildProcessProxy No tests found, exiting with code 0

13:13:47 (1428) INFO DryRunExecutor No tests were found
13:13:47 (1428) DEBUG Stryker Not removing the temp dir because an error occurred
13:13:47 (1428) ERROR Stryker No tests were executed. Stryker will exit prematurely. Please check your configuration.

So I cannot understand why it doesn't find any test, since it looks like the jest configuration file is read properly

Stryker environment

I have the following Stryker installed in my package.json:

@stryker-mutator/jest-runner": "^8.6.0",
"@stryker-mutator/core": "^8.6.0",
"@stryker-mutator/typescript-checker": "^8.6.0",
"jest": "^29.7.0",

and my stryker.config.mjs contains this:

const config = {
  _comment:
    "This config was generated using 'stryker init'. Please take a look at: https://stryker-mutator.io/docs/stryker-js/configuration/ for more information.",
  packageManager: "npm",
  reporters: ["html", "clear-text", "progress", "dashboard"],
  checkers : ["typescript"],
  tsconfigFile: 'tsconfig.json',

  testRunner: "jest",
  jest: {
    projectType : 'custom',
    configFile : 'jest.config.cjs',
    enableFindRelatedTests: true,
  },
  ignorePatterns : ["lib/*"],
  mutate : ['src/**/index.ts'],
  tempDirName : 'stryker-temp',
  disableTypeChecks: "app/**/*.{js,ts,jsx,tsx,html,vue}",
  testRunner_comment:
    "Take a look at https://stryker-mutator.io/docs/stryker-js/jest-runner for information about the jest plugin.",
  coverageAnalysis: "perTest",
  logLevel : 'trace',
};
export default config;

my jest.config.cjs contains this:

module.exports = {
  verbose: false,
  testMatch: ['<rootDir>/src/**/test.ts'],
  moduleFileExtensions: ['js', 'mjs', 'ts'],
  testPathIgnorePatterns: [
    '<rootDir>/index.index.index.index.index.index.index.index.index.index.index.index.index.index.test.js',
  ],
  transform: {
    '^.+\\.tsx?$': 'ts-jest',
    '^.+\\.jsx?$': 'babel-jest',
    '^.+\\.mjsx?$': 'babel-jest',
    '^.+\\.mjs$': 'babel-jest',
    "^.+\\.js$": "babel-jest"
  }
}

Additional context

I tried to turn the mjs striker config file into a cjs to verify any compatibility with ecmascript module, but it didn't work.

Here I attach the link of my project. Thanks, any help is appreciated 🤗

Fulcinator commented 5 days ago

Little update here:

I tried setting the mutate to skip my test, just to make sure it doesn't happen. I also modified the coverageAnlaysis parameter turning it off.by setting this:

mutate : ['src/**/index.ts', "!src/**/test.ts","!src/**/test.js"],
coverageAnalysis: "off",

I checked the temp folder, and it seems that mutants are properly generated and the tests are not mutated.

Now, running npx striker run the error output is the following:

13:13:36 (13868) ERROR DryRunExecutor One or more tests resulted in an error:
        Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23), Your test suite must contain at least one test. Error: Your test suite must contain at least one test.
    at onResult (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:133:18)
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\@jest\core\build\TestScheduler.js:254:19
    at C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:363:13
    at Array.map (<anonymous>)
    at Emittery.emit (C:\Users\d063684\Desktop\Esperimento\Project\date-experiment-2024-mutation\node_modules\emittery\index.js:361:23)
13:13:38 (13868) ERROR Stryker Unexpected error occurred while running Stryker Error: Something went wrong in the initial test run
    at DryRunExecutor.validateResultCompleted (file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/core/dist/src/process/3-dry-run-executor.js:76:15)
    at DryRunExecutor.executeDryRun (file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/core/dist/src/process/3-dry-run-executor.js:96:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async WorkItem.execute (file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/core/dist/src/concurrent/pool.js:32:28)
    at async file:///C:/Users/d063684/Desktop/Esperimento/Project/date-experiment-2024-mutation/node_modules/@stryker-mutator/core/dist/src/concurrent/pool.js:69:13

Any support is very appreciated. Thanks 🤗