Some of the optimisations that Dredd does to avoid redundant or equivalent mutants are fiddly. It would be good to have at least some exhaustive tests to check for these. What I'm thinking about is something like:
a single-file example
an "expectations" file, comprising a series of Python regular expressions, one per line
For each such test, the following should be done:
Dredd is applied to the single-file example
The example is compiled with sanitizers (ubsan and asan)
The example is executed with no mutations enabled, and it is confirmed that its standard output matches the first regex
The example is executed again and again with each possible mutant, and it is confirmed that:
For each mutant, at least one regex is matched, either by standard output or standard error (the latter so that mutants that will lead to UB can be checked)
All the regexes are matched at least once, across all of the mutants
This will allow us to check, for example, that optimisations are not leading to a loss of coverage.
Some of the optimisations that Dredd does to avoid redundant or equivalent mutants are fiddly. It would be good to have at least some exhaustive tests to check for these. What I'm thinking about is something like:
For each such test, the following should be done:
This will allow us to check, for example, that optimisations are not leading to a loss of coverage.