Failed to build the mutated assembly due to unrecoverable error: error CS0518: Predefined type 'System.Attribute' is not defined or imported

[NanoRossi]

Describe the bug The above error occurs when running dotnet stryker with one of my test projects,

Logs

[10:50:30 INF] Started initial build using dotnet build [10:50:41 INF] Initial build successful [10:51:03 INF] Using testrunner VsTest [10:51:03 INF] Total number of tests found: 57 [10:51:03 INF] Initial testrun started [10:51:06 INF] Using 9219 ms as testrun timeout [10:51:11 ERR] Failed to build the mutated assembly due to unrecoverable error: error CS0518: Predefined type 'System.Attribute' is not defined or imported [10:51:11 ERR] An error occurred during the mutation test run

General Build Failure detected.

[10:51:11 INF] Time Elapsed 00:00:50.3424653

[rouke-broersma]

Could you add the additional info asked in the bug template? Like what dotnet version you're using and what stryker version you're using?

[rouke-broersma]

I'm gonna go ahead and close this until you provide the above information. Feel free to reply and I'll re-open the issue.

[Rjae]

Stryker: 0.16.1 (beta) .NET Core: 3.1.101

I get this for previously successful builds. For example... Succeeded: https://github.com/appshapes-org/logging-console/actions/runs/84933566 Failed: https://github.com/appshapes-org/logging-console/actions/runs/84981650

[rouke-broersma]

@Rjae Awesome, thanks!

So does this happen intermittently at random or is there any kind of pattern you can detect? Does this only happen in the build or also on your own systems?

Also, you are some versions behind the latest, is this on purpose? If not, could you try updating to 0.17.1 and see if it improves anything?

[Rjae]

It does happen locally as well. I have not tried to dig in to why yet, but I will try a version bump this morning and report results here. Thanks!

[Rjae]

@Mobrockers the version upgrade seems to have resolved the issue. Thanks!

[rouke-broersma]

Cool! If you happen to notice that it's still failing sometimes please let us know

[MaurizioPz]

I'm having the same problem on Stryker 0.18.0 The project is an NUnit test project and the main one is .NET Framework v4.6.1 (Windows)

These are the logs

[15:00:53 INF] Identifying project to mutate. [15:01:04 INF] The project will be mutated. [15:01:14 INF] Analysis complete. [15:01:14 INF] Building test project (1/1) [15:02:06 INF] Initializing test runners (VsTest) [15:02:18 INF] Test runners are ready [15:02:18 INF] Total number of tests found: 11 [15:02:18 INF] Initial testrun started [15:02:38 INF] Using 34769 ms as testrun timeout [15:02:43 ERR] Failed to build the mutated assembly due to unrecoverable error: error CS0518: Predefined type 'System.Attribute' is not defined or imported [15:02:43 ERR] An error occurred during the mutation test run

General Build Failure detected.

[15:02:43 INF] Time Elapsed 00:01:49.7608697

[rouke-broersma]

@MaurizioPz

So does this happen intermittently at random or is there any kind of pattern you can detect?

Does this only happen in the build or also on your own systems?

Could you provide us with a repro? At this point we cannot investigate the issue as we have not experienced it and don't know when it happens.

[AlissonPereiraNeves]

Hi, I have this same problem, but intermittently and only in the execution that occurs in the pipeline (Jenkins). Stryker 0.18.0. .NET Core 2.0

I have multiple projects in my solution, and the problem always occurs when executing the tests for the same project.

Logs: Version: 0.18.0 (beta)

[16:51:03 WRN] Using 4 testrunners which is more than reccomended 1 for normal system operation. This can have an impact on performance. [16:51:04 INF] Identifying project to mutate. [16:51:05 INF] The project /home/jenkins/agent/workspace/ will be mutated. [16:51:07 INF] Analysis complete. [16:51:07 INF] Building test project /home/jenkins/agent/workspace (1/1) [16:51:09 INF] Initializing test runners (VsTest) [16:51:12 INF] Test runners are ready [16:51:12 INF] Total number of tests found: 11 [16:51:12 INF] Initial testrun started [16:51:14 INF] Using 12248 ms as testrun timeout [16:51:16 ERR] Failed to build the mutated assembly due to unrecoverable error: error CS0518: Predefined type 'System.Attribute' is not defined or imported [16:51:16 ERR] An error occurred during the mutation test run

General Build Failure detected.

[rouke-broersma]

@NanoRossi @MaurizioPz @AlissonPereiraNeves @Rjae

Could all of you let us know which OS you're building on?

[Rjae]

Windows 10 and a couple Linux distros. No issues here for a long time now.

[AlissonPereiraNeves]

Windows 10 and Linux (Ubuntu).

In Windows builds (on my local machine) I don't have the problem. In Linux builds that occur in the pipeline step, the problem happens intermittently.

[rouke-broersma]

Thanks for the confirmation! When I google this issue I find only mentions of Linux with mono (full framework linux) and dotnet core, no windows issues.

@AlissonPereiraNeves Do you have one Jenkins build agent or do you have multiple? And if you have multiple, can you make sure that you have the exact dotnet core version mentioned in your project installed on every single Jenkins agent?

[AlissonPereiraNeves]

We have multiples and, yes, we are sure that we have the exact dotnet version installed in all.

[ghost]

@Mobrockers We are also getting this error unfortunately We are running in docker

FROM golang:1.15.2-buster
apt-get install -y dotnet-sdk-3.1
dotnet tool install -g dotnet-stryker

We are running stryker multiple times for different projects. Happy to go for a different docker image if you think one would work better.

[rouke-broersma]

Hi @jdwalker-emis I'm sorry I have no suggestions for you, we have no clue what causes this issue and have no leads to investigate.

[rouke-broersma]

All that have had this problem, our upstream (Buildalyzer) has recently improved some similar design-time build target issues for certain project types. I would love to hear from you guys if you still experience this problem.

[izzy32457]

I have just come across this issue. I believe I am experiencing it on my local Windows 10 laptop. Current version of the Stryker tool is 1.3.1

I appear to get a ton of these errors and they all appear to be things missing from System namespace libraries.

If needed I could share my "trace" log file and any other project files if it will help?

[soundaryakumar073]

I'm facing the same issue in my local. Current Stryker version: 1.5.2, .NET version: NET6.0. Any updates on this issue?

[izzycoding]

Not sure if this will help, but after a lot of playing about I ended up with a stryker-config.json file that seems to work on my project.

{
    "stryker-config":
    {
        "target-framework": "net6.0",
        "solution": "Relative/Path/To/Solution.sln",
        "reporters": [ "html", "progress" ],
        "coverage-analysis": "off",
        "ignore-methods": ["ConfigureAwait", "*Exception.ctor", "Console.Write*", "Debug.*"]
    }
}

I have found that the combination of the "solution" and "coverage-analysis" is what appears to fix this issue for me.

[soundaryakumar073]

My solution had many projects within separate folders re-organizing them with just the project and test project without nesting inside folders fixed the issue.

[rouke-broersma]

That's interesting

[chivince]

I also run into this problem.

My project:

• targets the framework netcoreapp2.1
• is built with the .NET Core SDK 6.0.200
• uses Stryker-net 3.1.0

However, the same project with Stryker-net 3.0.1 worked totally fine.

[JackSteel97]

I'm also seeing this problem 100% of the time when running Stryker against a .NET 6 app in Azure Pipelines using Stryker Version 3.10.0 but never locally

[rouke-broersma]

It tends to happen more in cicd pipelines for some reason, we don't know why unfortunately.

[JackSteel97]

It tends to happen more in cicd pipelines for some reason, we don't know why unfortunately.

Is there any effort being put into resolving this here or is this purely an issue for Buildalyzer? This is currently blocking our mutation testing automation implementation because it happens 100% of the time on 100% of our .NET API projects

[rouke-broersma]

@JackSteel97 afaik we have zero reproductions so far. So if you are able to so reliable replicate this behavior we would love to receive a broken example from you. When we have it we can see if this is something we can fix in Stryker, in buildalyzer or not.

[JackSteel97]

Unfortunately, I also haven't been able to reproduce it with a minimal example - only entire work projects which obviously I can't share. I have found that sometimes disabling mix mutants fixes it, and ignoring files it mentions in the compilation errors via the mutate config also works but I assume that's just because it doesn't do anything instead of whatever is breaking so potentially not very helpful. I'll keep adding stuff to my minimal reproduction project and trying to get it to break and let you know if I manage it

[dupdob]

@JackSteel97 : disabling mutant should have no effect whatsoever as it impacts only the test phase.

We know of one (random/transient) cause for this problem: sometimes Stryker concludes the project has no reference assembly whatsoever. We use Buildalyzer for this, and the working assumption is that nuget retrieval times out, resulting in a failed analysis.

This is transient as it disappears once the cache have been populated. But it can be recurring on a CI pipeline if the cache is cleared every time or forcibly refreshed.

I a mentioning this because none of your workaround should have any impact on this, so there may be something else we are not aware of.

[JackSteel97]

@dupdob There must be something else going on then because disabling mutations on particular files that throw the compilation errors (not the System.Attribute error but similar ones about stuff like System.Int32) definitely fixes it. Getting the mutation config sorted I can reliably go from failing with the same error 100% of the time to succeeding 100% of the time.

Thanks for the additional context I'll have a play around with the nuget caching whenever I get the chance to.

[mu88]

Just for the sake of completeness: I'm still seeing this error from time to time both locally and in our CI environment (using Stryker v4.0.3):

Logs

``` [15:51:28 INF] Analysis starting. [15:51:31 INF] Found 3 source projects [15:51:31 INF] Found 1 test projects [15:51:31 WRN] Buildalyzer could not find sourcefiles. This should not happen. Will fallback to filesystem scan. Please report an issue at github. [15:51:32 INF] Found project C:\work\Bitbucket\MyOrg\my-app\src\MyCompany.MyApp.Features\MyCompany.MyApp.Features.csproj to mutate. [15:51:32 INF] Analysis complete. [15:51:32 INF] Building solution Stryker.sln [15:51:50 INF] Number of tests found: 922 for project C:\work\Bitbucket\MyOrg\my-app\src\MyCompany.MyApp.Features\MyCompany.MyApp.Features.csproj. Initial test run started. [15:51:56 ERR] Failed to build the mutated assembly due to unrecoverable error: error CS0518: Predefined type 'System.Attribute' is not defined or imported [15:51:56 ERR] An error occurred during the mutation test run General Build Failure detected. [15:51:56 INF] Time Elapsed 00:00:28.8792810 Unhandled exception. General Build Failure detected. ```

However, I cannot reproduce it reliably, too :/

[renzorodrigues]

This worked for me: I already have dotnet-tools.json file on my project, so I don't need to execute dotnet new tool-manifest in my pipeline. I just commented this line and Stryker worked as expected.

[dupdob]

I can share some hindsight here: this is due to the project analysis failing to identify ANY assembly reference. We use Buildalyzer to perfom the analysis, and it needs to perform a clean (simulated) build to analyze everything. The 'clean' part is required to identify the dependencies (I can't explain why, I do not fully understand Buildalyzer yet). If this fails for some reason, no dependency is identified and Stryker will fail to build the project. This may be due to some concurrent building: the project is being built by some other step of the pipeline while the analysis is in progress.

We should improve on error reporting (lack of dependency is a strong hint), but this would not get us any closer to a real fix