Experiment mutation testing

[ArnaudBuchholz]

Provided the fact that coverage is split per journey (and qUnit), it would be possible to optimize mutation testing by knowing which tests to execute depending on which file is altered.

What needs to be done :

• [ ] Compute file / test dependency (might go down to lines to be efficient)
• [ ] Select an AST parser for javascript
• [ ] Provide a syntax for mutation description (what to match / what to do)
• [ ] Build the list of possible mutation, quantified by : number of tests to run / number of times it is covered ...
• [ ] Determine parameters for mutation selection (all, some, most relevant, fastest...)
• [ ] Run the mutations and report on the undetected mutations

[ArnaudBuchholz]

Example of possible mutation : x < y ⇒ x > y 0 ⇒ 1 new Promise(function (resolve, reject) { ... }) ⇒ new Promise(function (reject, resolve) { ... }) any function call which result is not reused, remove the function call

[ArnaudBuchholz]

Generate a mutation score : #killed mutants / # non equivalent mutants

Equivalent mutants generates the same output

[ArnaudBuchholz]

https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/

[ArnaudBuchholz]

Prioritize qUnit tests over OPA tests. If a mutation can be killed with a QUnit test, simply ignore the OPA ones. test timeout means mutant should get killed. It would be interesting to keep track of test duration to determine a valid timeout.

[ArnaudBuchholz]

Closing for now, not the top priority