Mocha-test-runner executes tests that should not be executed

[mcdr2k]

Summary

Mocha-test-runner may execute tests for a mutant that do not cover that mutant (even though coverageAnalaysis is enabled). Assume that the test titled 'some test' covers mutant x. Then, at the moment, a test titled 'some test that does not cover x' is also executed. In fact, every test that starts with the string 'some test' is executed.

It is unclear whether other test-runners have the same issue.

Cause & possible Fix caused by mocha-test-runner: const metaRegExp = testFilter.map((testId) => `(${escapeRegExp(testId)})`).join('|');

This can be fixed by appending a dollar sign after the escaped regex: const metaRegExp = testFilter.map((testId) => `(${escapeRegExp(testId) + '$'})`).join('|');

Stryker config

This bug was found during development with stryker-js, meaning that all configurations are the same as the current master branch of stryker-js.

[nicojs]

Great find 👏

How about the start of the regex? I would expect a ^ as well as a $.

[mcdr2k]

Yes you are right, both caret and dollar sign are necessary. The possible fix for mocha-test-runner would then be: const metaRegExp = testFilter.map((testId) => `(^${escapeRegExp(testId)}$)`).join('|');

[nicojs]

Did you want to fix this one @mcdr2k ?

[mcdr2k]

Yes, I would like to fix this one. However, at the moment I am investigating an issue that could be related to this.