stryker-mutator / stryker-js

Mutation testing for JavaScript and friends
https://stryker-mutator.io
Apache License 2.0
2.6k stars 251 forks source link

Mutation coverage #1910

Closed wolf-off closed 4 years ago

wolf-off commented 4 years ago

Problem It's hard to analise failed mutants, when they are just list.

Possible solution Use mechanizm like in code-coverage:

  1. Green by default
  2. Red block/operator/etc. if mutation of it is alive

Benefit It's allow:

  1. To visualise place with bad mutation coverage in files.
  2. Generate statistic like in code coverage by operators (may be lines).

Analog In main page of PiTest there are information about it, but i have not check it https://pitest.org/

simondel commented 4 years ago

Do you mean a HTML report like this? https://dashboard.stryker-mutator.io/reports/github.com/stryker-mutator/stryker4s/master?module=core

wolf-off commented 4 years ago

My knowledge about project out of date. Is it planed to add some calculation by lines?

Problem Block mutant with 30 lines of code is alife, and other 30 lines have 30 killed mutants It is 97% killed mutants, but half of file have not tested

simondel commented 4 years ago

Ah I know what you mean. You don't just want to know what the percentage of killed mutants is, but you also want to know how many lines of code contain tested mutants.

What do you expect when a line of code has multiple mutations and some of them are killed and some are not killed?

wolf-off commented 4 years ago

Think It shoul be red if some mutation change/delete this satement/operator/function/unit/etc and alive Example with ConditionalExpression - we can mutate it to 'true' and 'false'. If one alife, it is bad tested place - paint by red. If both have been killed, but some parend block mutated by BlockStatement alife - all block should be painted by red. Further we can use common behaviour - if line contain any red statement - it is not covered

simondel commented 4 years ago

Most of the things you suggest are already implemented in the HTML reporter. We plan on staying at that functionality for now and won't add support for calculating the risk of the mutations that survived in the application.