stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.76k stars 176 forks source link

Extend information about survived mutant with keyword used to disable mutant in coment #2460

Open CzBuCHi opened 1 year ago

CzBuCHi commented 1 year ago

Is your feature request related to a problem? Please describe. When mutant survives html report will show bottom panel with explanation why. But it doesnt show how to disable that particular mutant.

For example i++ mutates to i-- and survives and i get this message:

PostIncrementExpression to PostDecrementExpression mutation Survived (74:36)

Describe the solution you'd like i would like to see there what comment i need to write in source code to disable given mutation - something like "to disable mutant write \\ Stryker disable once Update coment before mutated code" (probably right under 'mutation Survived' line -or- directly in source code, but that seems too complicated ...)

Describe alternatives you've considered Currently i need to search for https://stryker-mutator.io/docs/stryker-net/mutations/ page (because i dont remember its url) to find out that i need to use word update in \\ Stryker disable comment to disable this mutant

rouke-broersma commented 1 year ago

Interesting request but that kinda feels like we recommend ignoring all survived mutants, which is the opposite of what we would recommend.

CzBuCHi commented 1 year ago

good point ... so maybe reduce it only to Update: PostIncrementExpression to PostDecrementExpression mutation Survived (74:36)

-or-

add menu for https://stryker-mutator.io/docs/stryker-net/*/ pages into html report

rouke-broersma commented 1 year ago

Yep, we should probably start listing the mutator group name as well, good idea.

Liam-Rougoor commented 1 year ago

If we were to add suggestions, I would rather see explanations of how to kill the mutant than of how to ignore it.

rouke-broersma commented 1 year ago

Good luck with that :p

CzBuCHi commented 1 year ago

If we were to add suggestions, I would rather see explanations of how to kill the mutant than of how to ignore it.

ok ... show me how to kill these mutants:

int Something(int a, int b){
   if (a == 0) {
     return 0; // mutant 1: statement removed
   }
   if (b == 0) {
     return 0; // mutant 2: statement removed
   }
   return a * b; // expensive operation here - say http request to server
}

because i have no idea how to do it ...

dupdob commented 2 months ago

I think this a valuable feature request, I would rephrase it as: provide better help on how to ignore specific mutations. The big question being how and where: it should be easy, but not so easy people choose to ignore mutations instead of finding how to kill them.

Reverting to your example:

int Something(int a, int b){
   if (a == 0) {
     return 0; // mutant 1: statement removed
   }
   if (b == 0) {
     return 0; // mutant 2: statement removed
   }
   return a * b; // expensive operation here - say http request to server
}

My understanding is that this method contains optimizations to skip making a useless and costly method/api call. You have two approaches to deal with that:

1. Using spies (or mocks) Assuming this call happens through an interface (or an interface can be introduced), you can mock this interface Code loosely based on Moq:

[Test]
public ShouldSkipCall()
{
  ICostlyEndpoint mock = new Mock<ICostlyEndPoint>();

  var sut = new TestedComponent(mock.Object);
  // should skip the call
  sut.SomeThing(0,1);
  // verify we do make the expensive call
  mock.Verify( t => t.ExpensiveCall(...), Times.Never());
}

2.Measure execution time Only strategy I can think of if you can't mock the expensive call.

[Test]
public ShouldSkipCall()
{
  var sut = new TestedComponent();
  var timer = new StopWatch();
  // should skip the call
  sut.SomeThing(0,1);
  // verify we do make the expensive call
  timer.Stop();
  // let's say the expensive call lasts at least 10 milliseconds
  timer.TotalMilliseconds.ShouldBeLessThan(10);
}

Hope this helps, even if late

PS: I am currently reviewing and reassessing all open issues