stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.55k stars 242 forks source link

Different scores at each run in Angular 17 project #4878

Open rdechiara opened 4 weeks ago

rdechiara commented 4 weeks ago

Summary

Almost each time I run Stryker against my Angular v17 project I get different score without changing the code. I have noticed this issue because after getting a low score for a component I added the needed tests and the score went up. When I run the tests again, without any modification, the score dropped by about 20%. Stryker config

// @ts-check
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
const config = {
  _comment:
    "This config was generated using 'stryker init'. Please see the guide for more information: https://stryker-mutator.io/docs/stryker-js/guides/angular",
  mutate: [
    "src/**/*.component.ts",
    "src/**/*.service.ts",
    "src/**/*.interceptor.ts",
    "!src/**/*.spec.ts",
    "!src/test.ts",
    "!src/environments/*.ts",
  ],
  testRunner: "karma",
  karma: {
    configFile: "karma.conf.js",
    projectType: "angular-cli",
    config: {
      browsers: ["ChromeHeadless"],
    },
  },
  reporters: ["progress", "clear-text", "html"],
  concurrency: 4,
  concurrency_comment:
    "Recommended to use about half of your available cores when running stryker with angular",
  coverageAnalysis: "perTest",
};
export default config;

Test runner config

Stryker environment

+-- @stryker-mutator/core@8.2.2
+-- @stryker-mutator/karma-runner@82.2
+-- karma@6.4.3

Test runner environment

npx stryker run

Didn't configure karma so it has all defaults.

Your Environment

software version(s)
node 18.19.0
npm npm 10.8
Operating System Windows 11

Add stryker.log