stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.75k stars 176 forks source link

Add command line option for opening report only when mutant survives / close successful report #2667

Open CzBuCHi opened 10 months ago

CzBuCHi commented 10 months ago

Is your feature request related to a problem? Please describe. I would like to open html report only when mutant survive, which is currenly not possible.

Describe the solution you'd like open html report only when mutant survive -or- auto close report when all mutants are killed

Describe alternatives you've considered

  1. Return non-zero exit code so i can do IF NOT ERRORLEVEL 0 OPEN_REPORT in command line
  2. modify --open-report flag to allow something like --open-report=close-successful
    • deserialized as enum (true, false, close) instead of boolean
    • current behavior still apply (for backwards compatibility)
    • when new version used and all mutants are killed report will automatically close

EDIT: 3. --open-report=when-failed that would open report after first mutant survives

rouke-broersma commented 10 months ago

Modifying auto open sounds good, I like it. But I don't think auto close is possible. How would we instruct the browser to do that?

CzBuCHi commented 10 months ago

How would we instruct the browser to do that?

my first tought was taskkill /pid 1234 but now when im thinking about it more is see the issue (browser will not close tab) ...

rouke-broersma commented 10 months ago

How would we instruct the browser to do that?

my first tought was taskkill /pid 1234 but now when im thinking about it more is see the issue (browser will not close tab) ...

Well 1 we wouldn't have permission to do that and 2 we wouldn't know what the pid is because we have no way of knowing which browser is is even hosting the report.

CzBuCHi commented 10 months ago

got another idea: return number of survived mutants to caller (this one is probably the simplest to implement) - so i can do something like:

dotnet stryker ...
if not errorlevel 0 open_report
richardwerkman commented 10 months ago

I don't see much value in this, to be honest. Why do you actually want this? What about building a notification system in the report? So that you get notified when a surviving mutant is found.

CzBuCHi commented 10 months ago

Why do you actually want this?

im running multiple stryker call in batch file (to save time i run stryker only whn report file is outdated) and currently to find out if mutants survived i need: 1) keep watching console 2) use -o parameter to open html report -or- open report manually afterwards

in both situations i end up with report opened in browser even when no mutants survived ... i would prefer to open report only when mutants are found

What about building a notification system in the report?

notify who? and how?