Same configuration in different react apps is causing failure in mutation tests at all or finds just few tests

[Vaclovas]

Question

Why same configuration for a very similar app is causing test failure? Mainly, tests are not even running or only a few tests can be found.

Stryker environment

stryker config: { "reporters": [ "html", "clear-text", "progress" ], "packageManager": "npm", "timeoutMS": 30000, "ignorePatterns": ["/src/lib/icons/*.tsx"], "tsconfigFile": "tsconfig.json", "mutator": { "excludedMutations": ["BlockStatement", "StringLiteral", "ArrayDeclaration", "ObjectLiteral"] }, "disableTypeChecks": "*/.{js,ts,jsx,tsx,html,vue}", "testRunner": "jest", "coverageAnalysis": "perTest", "tempDirName": "stryker-tmp", "mutate": [ "/src///*.ts", "/src///.tsx", "!/src//.spec.ts", "!/src/*/.spec.tsx" ] }

nx + jest+ react and "@stryker-mutator/core": "8.2.6", "@stryker-mutator/jest-runner": "8.2.6", "jest": "29.4.1", "jest-environment-jsdom": "29.4.1", "jest-environment-node": "29.4.1", "jest-slow-test-reporter": "1.0.0",

Additional context

Here is the first app:

here is another app: What is intresting, that even components are covered by unit tests, they are marked as "NoCoverage", jest finds these tests and succesfully adds those to coverage report, but stryker - is failing. Assumption can be, that configuration is bad, but some of the files are found and counted into the report: So, actually, only 6 spec.ts/tsx files are identified from 256 files. I've tried to search for differences, remove specific imports, change naming - nothing happens, tried some recipies from github and troubleshooting guide.

Maybe you could assist with this?

[Vaclovas]

ok, seems like I've found the issue which is skipoing those tests - this is aliased imports... Can you provide info, why those are not working?

[nicojs]

Hmm interesting. I think this is an issue with Jest. Could you try running Stryker with this change in your config?

{
  "reporters": ["html", "clear-text", "progress"],
  "packageManager": "npm",
  "timeoutMS": 30000,
  "ignorePatterns": ["/src/lib/icons/*.tsx"],
  "tsconfigFile": "tsconfig.json",
  "mutator": { "excludedMutations": ["BlockStatement", "StringLiteral", "ArrayDeclaration", "ObjectLiteral"] },
  "disableTypeChecks": "/.{js,ts,jsx,tsx,html,vue}",
  "testRunner": "jest",
  "coverageAnalysis": "perTest",
  "tempDirName": "stryker-tmp",
-  "mutate": ["/src//**/.ts", "/src///*.tsx", "!/src//*.spec.ts", "!/src/**/*.spec.tsx"]
+  "mutate": ["/src//**/.ts", "/src///*.tsx", "!/src//*.spec.ts", "!/src/**/*.spec.tsx"],
+  "jest": {
+    "enableFindRelatedTests": false
+  }
}

[Vaclovas]

Hmm interesting. I think this is an issue with Jest. Could you try running Stryker with this change in your config?

{
  "reporters": ["html", "clear-text", "progress"],
  "packageManager": "npm",
  "timeoutMS": 30000,
  "ignorePatterns": ["/src/lib/icons/*.tsx"],
  "tsconfigFile": "tsconfig.json",
  "mutator": { "excludedMutations": ["BlockStatement", "StringLiteral", "ArrayDeclaration", "ObjectLiteral"] },
  "disableTypeChecks": "/.{js,ts,jsx,tsx,html,vue}",
  "testRunner": "jest",
  "coverageAnalysis": "perTest",
  "tempDirName": "stryker-tmp",
-  "mutate": ["/src//**/.ts", "/src///*.tsx", "!/src//*.spec.ts", "!/src/**/*.spec.tsx"]
+  "mutate": ["/src//**/.ts", "/src///*.tsx", "!/src//*.spec.ts", "!/src/**/*.spec.tsx"],
+  "jest": {
+    "enableFindRelatedTests": false
+  }
}

1. Jest is running successfully and tests are being identified.
2. After the configuration change, all tests can be seen in the tests tab, but all cases survived, none were identified as killed.
3. If I'm changing aliased import to relate one - Stryker is able to find a test and killed cases ar identified.
4. 4. Running time increased by a huge amount.

[nicojs]

This sounds like this known issue: https://stryker-mutator.io/docs/stryker-js/troubleshooting/#all-mutants-survive---jest-runner

Could you try enabling --inPlace? That should do the trick. Note: using --inPlace doesn't allow you to change your code when StrykerJS is running.

[Vaclovas]

Hello, --inPlace is not changing any results. I've created a project boilerplate for this: project.zip open project, run "npm i" inside the root, after that continue the same on shell and "ui-shared" folders. Run "npx stryker run" with changing FieldLabel.spec.tsx file. Comment/uncomment import line for the same file import, and see how results are changing:

report difference: