stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.56k stars 241 forks source link

Implementation of mutation levels #4686

Open Ja4pp opened 6 months ago

Ja4pp commented 6 months ago

What?

We've added support for mutation levels to improve the performance of Stryker. It includes mutation operators, mutation groups, mutation levels, documentation and tests. In addition, we have PRs for the reporters in stryker-mutator/mutation-testing-elements and stryker-mutator/stryker-dashboard. Those PR's are still a work in progress.

Why?

Currently, Stryker, by default, executes a comprehensive mutation testing suite on your project, offering maximum efficacy. However, this default setting demands substantial computational resources. Alternatively, users can leverage mutation levels for a more resource-efficient testing experience, ranging from level 1 (optimized for speed) to level 3 (prioritizing efficacy). For more background information, see the Calisto research (https://research.infosupport.com/callisto/).

How?

For a tailored configuration, Xavier allows the adjustment of mutation levels by specifying inclusion and exclusion fields. In the configuration file, mutation operators can be precisely designated by their literal names, while mutator groups and mutation levels are denoted with the @ prefix (e.g., @ArithmeticOperator or @level1). For instance, to refine level 2 by selectively retaining the AdditionOperatorNegation mutation operator and excluding all other operators within the ArithmeticOperator group, the configuration would be expressed as follows:

{
   "includedMutators": ["@level1", "AdditionOperatorNegation"],
   "excludedMutators": ["@ArithmeticOperator"]
}

Testing?

We have made modifications to the ignore-project in the e2e test folder to support mutation levels as it had excluded mutations already in its configuration file. Moreover, we added unit tests for all mutator operators. We tested a subset of operators, an empty level and an undefined level.

dvcopae commented 5 months ago

Changes since Nico's last review: