search

stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.78k stars 188 forks source link

Stryker fails with "Internal error due to compile error" when we reference a NuGet called MimeTypes #2930

Closed MilleBo closed 5 months ago

MilleBo commented 6 months ago

Describe the bug I'm currently attempting to run Stryker in one of our legacy solutions that utilizes a NuGet package called MimeTypes. However, for some reason, Stryker is unable to compile it and throws compilation errors such as the following:

The name 'MimeTypes' does not exist in the current context (Source code: MimeTypes)

To verify this issue, I created a fresh project, referenced MimeTypes, and added a sample class that primarily utilizes MimeTypes, like this:

namespace MimeTypeTest;

public class Class1
{
    public int DoThings()
    {
        MimeTypes.GetMimeType("fdsf.txt");

        var i = 1 + 2;
        return i;
    }
}

I can build it just fine from visual studio but If I run Stryker against the solution that only contains this class (and a simple unit test that calls this class), I get the following error in my logs:

[09:26:06 WRN] Stryker.NET encountered a compile error in C:\temp\MimeTypeTest\MimeTypeTest\Class1.cs (at 6:8) with message: The name 'MimeTypes' does not exist in the current context (Source code: MimeTypes)
[09:26:06 INF] Safe Mode! Stryker will flag mutations in DoThings as compile error.
[09:26:06 FTL] Stryker.NET could not compile the project after mutation. This is probably an error for Stryker.NET and not your project. Please report this issue on github with the previous error message.
[09:26:06 ERR] An error occurred during the mutation test run
System.AggregateException: One or more errors occurred. (Internal error due to compile error.)
 --->

Internal error due to compile error.

   --- End of inner exception stack trace ---
   at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
   at System.Threading.Tasks.Parallel.ForWorker[TLocal,TInt](TInt fromInclusive, TInt toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.ForWorker[TLocal,TInt](TInt fromInclusive, TInt toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters, ITestRunner runner) in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectOrchestrator.cs:line 69
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in /_/src/Stryker.Core/Stryker.Core/StrykerRunner.cs:line 63
[09:26:06 INF] Time Elapsed 00:00:11.8175394
Unhandled exception. System.AggregateException: One or more errors occurred. (Internal error due to compile error.)
 --->

Internal error due to compile error.

   --- End of inner exception stack trace ---
   at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
   at System.Threading.Tasks.Parallel.ForWorker[TLocal,TInt](TInt fromInclusive, TInt toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.ForWorker[TLocal,TInt](TInt fromInclusive, TInt toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters, ITestRunner runner) in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectOrchestrator.cs:line 69
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in /_/src/Stryker.Core/Stryker.Core/StrykerRunner.cs:line 63
   at Stryker.CLI.StrykerCli.RunStryker(IStrykerInputs inputs) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 102
   at Stryker.CLI.StrykerCli.<>c__DisplayClass11_0.<Run>b__0() in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 74
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass143_0.<OnExecute>b__0(CancellationToken _)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at Stryker.CLI.StrykerCli.Run(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 80
   at Stryker.CLI.Program.Main(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/Program.cs:line 14

So maybe someone with more knowledge about Stryker can tell me why this NuGet package creates issues, if there is a workaround, or if it's something that needs to be fixed in Stryker to handle?

Expected behavior It should compile the project just fine.

Desktop (please complete the following information):

dupdob commented 6 months ago

Thanks for your precise reporting. I reproduce the issue. I have yet to analyze what is the root cause, but MimeTypes is NOT listed as a reference by BuildAlyzer, and that is what Stryker uses. Update: this is because MimeTypes is not packaged as a library, but as a source file. This is a first for us. We need to understand how to detect this.

MilleBo commented 6 months ago

Thanks for your precise reporting. I reproduce the issue. I have yet to analyze what is the root cause, but MimeTypes is NOT listed as a reference by BuildAlyzer, and that is what Stryker uses. Update: this is because MimeTypes is not packaged as a library, but as a source file. This is a first for us. We need to understand how to detect this.

Thanks for looking into this and that you could reproduce it.

this is because MimeTypes is not packaged as a library, but as a source file

Interesting, think that's the first time I even heard about that type of packaging.

dupdob commented 6 months ago

Thanks. I have been able to work around the issue, but this is still kludgy. It still needs tests and refinment

dupdob commented 6 months ago

This turns to be harder than expected: the problem is that BuildAlyzer exhibits two problems:

  1. it fails to report source files from package's content.
  2. it clears any intermediate folder during project analysis, which deletes any such files

I have devised a work around for problem (1), but dealing with (2) requires a redesign of the project analysis logic so it can be done in two phases. In short, It requires time.

I am currently working on a redesign of the project analysis logic, so I will incorporate needed changes there, but there are still several weeks of work before it being done.

In the short time, a possible workaround is to reuse the MimeType.cs file directly in your project (as describled in the project readme).

rouke-broersma commented 6 months ago

I remember seeing this kind of nuget package before. Is this kind of package similar to #2345?

I think the conclusion there was that this works correctly on windows for some reason, perhaps we like learn something from that.

it clears any intermediate folder during project analysis, which deletes any such files

Are you talking about buildalyzer executing a clean during project analysis? In that case multiple attempts have been made to do that, none successful. The maintainers of buildalyzer state that without a clean there's a good chance analysis fails.

dupdob commented 6 months ago

remember seeing this kind of nuget package before. Is this kind of package similar to https://github.com/stryker-mutator/stryker-net/issues/2345?

It looks like a similar package, but I did not check it for confirmation?

I think the conclusion there was that this works correctly on windows for some reason, perhaps we like learn something from that.

It does not work correctly on windows, at least, not any longer, as I did reproduce on first trial. I suspect this relates to more or less subtle changes in the SDK. MSBuild default targets, to be more specific.

Are you talking about buildalyzer executing a clean during project analysis? In that case multiple attempts have been made to do that, none successful. The maintainers of buildalyzer state that without a clean there's a good chance analysis fails.

I did test removing clean from the default targets list. It made no change at all regarding the content files, plus Stryker failed for other reasons. At least, I learned something: Buildalyzer does not 'fail' without clean. That is, it does report success, but data is missing. For example, there is 0 (zero) reference identified. This leads to the infamous 'System.Attributes' (or some other class) is undefined.

What I have been able to achieve is:

  1. identifies the properties that describe the relevant folder (a sub folder of obj)
  2. grab any file that's lying there
  3. have a successful run

But I did by manually restoring files after analysis: at first, I thought this a configuration/version issue, because folder structure was still here. What I plan to do is have this specific step executed AFTER the build. That should do the trick. And open an issue on Buildalyzer, because I do not understand why some source files would not be listed.