stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.76k stars 175 forks source link

Problems with mutation of LINQ code - cannot implicitly convert type 'double[]' to 'int[]' #1533

Closed marcin-golebiowski closed 3 years ago

marcin-golebiowski commented 3 years ago

Describe the bug Stryker.NET is not able to mutate a method with some LINQ. All mutations are rollbacked.

using System;
using System.Collections.Generic;
using System.Linq;

namespace ExampleProject
{
    public class Test
    {
        public void SomeLinq()
        {
            var list = new List<List<double>>();
            int[] listProjected = list.Select(l => l.Count()).ToArray();
        }   
    }
}

Following mutant is created:

using System;
using System.Collections.Generic;
using System.Linq;

namespace ExampleProject
{
    public class Test
    {
        public void SomeLinq()
        {
            var list = new List<List<double>>();
            int[] listProjected = list.Select(l => (Stryker1bzhkKAubQNejLt.MutantControl.IsActive(0)?l.Sum():l.Count())).ToArray();
        }   
}

which fails compilation because double[] cannot be converted to int[].

Unit test with problem:

[Fact]
public void TestCompileMutant()
{
            var syntaxTree = CSharpSyntaxTree.ParseText(@"
using System;
using System.Collections.Generic;
using System.Linq;

namespace ExampleProject
{
    public class Test
    {
        public void SomeLinq()
        {
            var list = new List<List<double>>();
            int[] listProjected = list.Select(l => l.Count()).ToArray();
        }   
    }
}");
            var mutator = new CsharpMutantOrchestrator(options: new StrykerOptions(mutationLevel: MutationLevel.Complete.ToString()));

            var mutant = mutator.Mutate(syntaxTree.GetRoot());
            var references = new List<PortableExecutableReference>() {
                    MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
                    MetadataReference.CreateFromFile(typeof(List<string>).Assembly.Location),
                    MetadataReference.CreateFromFile(typeof(Enumerable).Assembly.Location),
                };

            Assembly.GetEntryAssembly().GetReferencedAssemblies().ToList().ForEach(a => references.Add(MetadataReference.CreateFromFile(Assembly.Load(a).Location)));

            var input = new MutationTestInput()
            {
                ProjectInfo = new ProjectInfo()
                {
                    ProjectUnderTestAnalyzerResult = TestHelper.SetupProjectAnalyzerResult(properties: new Dictionary<string, string>()
                        {
                            { "TargetDir", "" },
                            { "AssemblyName", "AssemblyName"},
                            { "TargetFileName", "TargetFileName.dll"},
                            { "SignAssembly", "true" },
                            { "AssemblyOriginatorKeyFile", Path.GetFullPath(Path.Combine("TestResources", "StrongNameKeyFile.snk")) }
                        },
                       projectFilePath: "TestResources").Object,
                    TestProjectAnalyzerResults = new List<IAnalyzerResult> { TestHelper.SetupProjectAnalyzerResult(properties: new Dictionary<string, string>()
                        {
                            { "AssemblyName", "AssemblyName"},
                        }).Object
                    }
                },
                AssemblyReferences = references
            };

            var rollbackProcess = new RollbackProcess();

            var target = new CompilingProcess(input, rollbackProcess);

            using (var ms = new MemoryStream())
            {
                var result = target.Compile(new Collection<SyntaxTree>() { mutant.SyntaxTree }, ms, null, false);
                result.RollbackResult.RollbackedIds.ShouldBeEmpty();
            }
        }
    }

Expected behavior A mutant can be compiled and has some mutation.

Desktop (please complete the following information):

dupdob commented 3 years ago

Thank you for the report and thank you for providing a way to reproduce this. Can you please clarify something for us: here you have a single mutation that will be rollbacked because it triggers compilation error. This is normal and an expected behavior: Stryker generates mutation that may lead to compilation errors, relying on the rollback logic to catch them up later.

So if your issue is having this mutation failing : Count() ⇒ Sum() ? ⇒ then please post a clarifying comments. Or does it triggers a removal of other mutants ? ⇒ then please update your sample to have a proper reproduction scenario.

Thanks for your help

marcin-golebiowski commented 3 years ago

I've reported this issue due to the error message when I was running Stryker.NET:

"Safe Mode! Stryker will try to continue by rolling back all mutations in method. This should not happen, please report this as an issue on github with the previous error message."

The better example will be following code:

using System;
using System.Collections.Generic;
using System.Linq;

namespace ExampleProject
{
    public class Test
    {
        public void SomeLinq()
        {
            var x = 1 + 2;
            var list = new List<List<double>>();
            int[] listProjected = list.Select(l => l.Count()).ToArray();
        }   
    }
}

And corresponding unit test that should pass (I hope I got this right):

[Fact]
public void TestCompileMutant()
{
            var syntaxTree = CSharpSyntaxTree.ParseText(@"
using System;
using System.Collections.Generic;
using System.Linq;

namespace ExampleProject
{
    public class Test
    {
        public void SomeLinq()
        {
            var x = 1 + 2;
            var list = new List<List<double>>();
            int[] listProjected = list.Select(l => l.Count()).ToArray();
        }   
    }
}");
            var mutator = new CsharpMutantOrchestrator(options: new StrykerOptions(mutationLevel: MutationLevel.Complete.ToString()));
            var mutant = mutator.Mutate(syntaxTree.GetRoot());

            var syntaxes = new List<SyntaxTree>();
            foreach (var helper in CodeInjection.MutantHelpers)
            {
                syntaxes.Add(CSharpSyntaxTree.ParseText(helper.Value, new CSharpParseOptions(languageVersion: LanguageVersion.Default),
                    helper.Key));
            }
            syntaxes.Add(mutant.SyntaxTree);

            var references = new List<PortableExecutableReference>() {
                    MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
                    MetadataReference.CreateFromFile(typeof(List<string>).Assembly.Location),
                    MetadataReference.CreateFromFile(typeof(Enumerable).Assembly.Location),
                    MetadataReference.CreateFromFile(typeof(System.IO.Pipes.PipeAccessRule).Assembly.Location),
                };

            Assembly.GetEntryAssembly().GetReferencedAssemblies().ToList().ForEach(a => references.Add(MetadataReference.CreateFromFile(Assembly.Load(a).Location)));

            var compiler = CSharpCompilation.Create("TestCompilation",
                syntaxTrees: syntaxes,
                options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary),
                references: references);

            using (var ms = new MemoryStream())
            {
                var compileResult = compiler.Emit(ms);
                compileResult.Success.ShouldBeFalse();
                var target = new RollbackProcess();

                var fixedCompilation = target.Start(compiler, compileResult.Diagnostics, false, false);
                var rollbackedResult = fixedCompilation.Compilation.Emit(ms);

                rollbackedResult.Success.ShouldBeTrue();

                Assert.Single(fixedCompilation.RollbackedIds);
            }
}