stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.56k stars 245 forks source link

Why Stryker-JS is slower compared to Pitest (Java mutation library) #4414

Open adarshnd14 opened 11 months ago

adarshnd14 commented 11 months ago

Question

Why Stryker-JS is slower compared to Pitest (Java mutation library)? We observe that for larger projects in Java, Pitest takes maximum around 30 min. But Stryker takes many hours to scan larger projects in JavaScript.

Stryker environment

+-- @stryker-mutator/core@4.6.0 with stryker-cli@1.0.1

Stryker config used

{
 packageManager: "npm",
 reporters: ["html", "clear-text", "progress", "json"] ,
 testRunne: "command",
 coverageAnalysis: "off",
 timeoutMS: 1000000
}
+-- jest@26.6.2 or above
dtorrensf commented 11 months ago

+1

nicojs commented 11 months ago

The way you have Stryker setup is not an apples-to-apples comparison. You're using the command test runner, which disables almost all performance enhancements that Stryker can use.

Since you're using Jest, you should use the @stryker-mutator/jest-runner. It supports almost all performance features, including per test coverage analysis.