search

stryker-mutator / mutation-testing-elements

🗃 A schema for mutation testing results with the web components to visualise it 📊
Apache License 2.0
28 stars 21 forks source link

Proposal: new mutation status for mutants that are covered by a test that fails even before mutation #158

Open rouke-broersma opened 5 years ago

rouke-broersma commented 5 years ago

In stryker-net we have coverage analysis so we can determine which mutants are covered by which tests. Currently we do not allow a mutation testrun if you have failing unit tests because this would mark every mutant as killed. With coverage analysis we can disable all unit tests that are failing while also marking and skipping all mutants covered by those failing unit tests. We would like to have a new status in the schema to reflect a mutant that is covered by a test but for which we cannot determine if it's killed or not due to the covering unit test not working.

rouke-broersma commented 4 years ago

@nicojs thoughts?

nicojs commented 4 years ago

Can we use the ignored status with this and reflect the reason in a (new) reason field?

rouke-broersma commented 4 years ago

Yes that's a good idea, we can fill the reason field already for others already as well.

nicojs commented 4 years ago

Great, let's do that than. Btw, I would suggest to have a compiler flag to disable this feature in Stryker, since you might be fooled by the mutation score.

rouke-broersma commented 4 years ago

Proposal: Ability to tag mutants with traits such as:

'Slow', 'Static context', 'Initial testrun failed' etc instead of new mutant statusses