stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.78k stars 188 forks source link

Ignoring constructors using ignore-methods does not ignore calls to base/this constructor #3007

Open nwalker-et opened 3 months ago

nwalker-et commented 3 months ago

Describe the bug I'd like to use ignore-methods to ignore calls to exception constructors, as I'm not interested in testing the contents of exception messages. This works when the exception message appears in a new Exception("...") expression, but not when the exception message appears in a call to base or this from another constructor.

Logs log-20240808.txt

image

Expected behavior Since base and this are both means of invoking constructors, I feel they should be ignored by the same settings that would ignore direct calls to those same constructors. For example, if my ignore-methods setting contains the string "Exception.ctor", then I expect Stryker to ignore calls to this from constructors of classes whose name matches the string "Exception", as well as calls to base from constructors of classes which extend classes whose name matches the string "*Exception".

Desktop (please complete the following information):

Additional context Here is the source code I've used to reproduce this issue:

`BaseConstructorIssueDemo.cs` ```csharp namespace LibraryProject; public class BaseConstructorIssueDemo { public void ThrowException1() { throw new SomeException("This is an exception message."); } public void ThrowException2() { throw new SomeException(false); } public void ThrowException3() { throw new SomeException(); } public void ThrowException4() { throw new Exception("Some text."); } } public class SomeException : Exception { public SomeException() : base("Default message") { } public SomeException(bool value) : this($"Value: {value}") { } public SomeException(string message) : base(message) { } } ``` `BaseConstructorIssueDemoTests.cs` ```csharp namespace LibraryProject.Test; public class BaseConstructorIssueDemoTests { private BaseConstructorIssueDemo exampleClass = new BaseConstructorIssueDemo(); [Fact] public void ThrowException1_ThrowsException() { Assert.ThrowsAny(exampleClass.ThrowException1); } [Fact] public void ThrowException2_ThrowsException() { Assert.ThrowsAny(exampleClass.ThrowException2); } [Fact] public void ThrowException3_ThrowsException() { Assert.ThrowsAny(exampleClass.ThrowException3); } [Fact] public void ThrowException4_ThrowsException() { Assert.ThrowsAny(exampleClass.ThrowException4); } } ``` `stryker-config.json` ```json { "stryker-config": { "ignore-methods": [ "*Exception.ctor" ] } } ```
dupdob commented 3 months ago

thanks for your detail reporting. Indeed, base call constructor are not considered as constructor invocation and it appears to be a bug. Before engaging into a fix, I would like to stress out that you will still need to add to ignore methods the constructor for SomeException. Are you comfortable with this? As a work around, you can use Stryker comments to disable mutations around constructors. You an use online documentation on how to use them.

nwalker-et commented 3 months ago

thanks for your detail reporting. Indeed, base call constructor are not considered as constructor invocation and it appears to be a bug. Before engaging into a fix, I would like to stress out that you will still need to add to ignore methods the constructor for SomeException. Are you comfortable with this? As a work around, you can use Stryker comments to disable mutations around constructors. You an use online documentation on how to use them.

I imagine that if I have an entry in ignore-methods like "Exception.ctor" (note the ), that any call to the constructors of SomeException, Exception, or any other class ending in "Exception" would all be ignored. I would assume that in the case of SomeException's constructors, any call to this would be ignored due to SomeException.ctor matching *Exception.ctor, and any call to base would be ignored due to the constructors of its base class (i.e., Exception.ctor) also matching the same pattern. Does that line up with your understanding, @dupdob?

dupdob commented 3 months ago

Yes, thank you. I just wanted to make it clear that there would be no class hierarchy search. this was identified as to be supported as well. That being said, fixing this requires redesigning significant parts of Stryker.