Score difference in single file scan and complete project scan

[adarshnd14]

Question

We observed that when we run mutation scan on a whole repository sometimes the score will get reduced when compared to the score we got when we run mutation scan on a single file of that repository.

Why we are getting the difference in the score?

Stryker environment

+-- @stryker-mutator/core@4.6.0 with stryker-cli@1.0.1

Stryker config used for running mutation scan on whole repository

{
 packageManager: "npm",
 reporters: ["html", "clear-text", "progress", "json"] ,
 testRunner" "command",
 coverageAnalysis: "off",
 timeoutMS: 1000000
}

Stryker config used for running mutation scan on one file of the repository

{
 packageManager: "npm",
 reporters: ["html", "clear-text", "progress", "json"] ,
 testRunner" "command",
 coverageAnalysis: "off",
 timeoutMS: 1000000,
 mutate: [
        "relative path of the file,
        "!{src,lib}/**/__tests__/**/*.js?(x)",
        "!{src,lib}/**/?(*.)+(spec|test).js?(x)",
        "!{src,lib}/**/*+(Spec|Test).js?(x)"
        ]
}

+-- jest@26.6.2 or above

[aturingmachine]

I am seeing a similar behavior with the vitest runner for Strkyer.

stryker-mutator/core version: 7.1.1 stryker-mutator/vitest-runner version: 7.1.1

Seems to happen on Vue files, where running a directory or entire codebase will produce results where 0 mutants are covered. However if I run coverage on just that one Vue file it will produce 100% coverage - by timing out each and every mutant. Also should be noted that manually setting the mutant in the source code causes the test to fail, which should mean the mutants are covered.

[nicojs]

@aturingmachine please report a separate issue for this and include all details. This sounds like a bug in the new @stryker-mutator/vitest-runner.

@adarshnd14 could you provide more details? Logs or a small reproduction example? Could you also try the latest version of Stryker? Why are you not using the @stryker-mutator/jest-runner plugin?