martincostello commented 10 months ago

Describe the bug

Stryker fails to compile code when run with the .NET 8 SDK.

Having updated Polly to use the .NET 8 SDK to build and adopt some C# 12 language syntax to resolve analyser warnings, our mutation tests that use Stryker are no longer working. This was working in earlier previews, but seems to have stopped working since RC1.

The latest attempt at upgrading with minimal changes to reduce the number of variables that could have broken things can be found here.

An older PR that's had various force pushes over the last few months can be found here, which contains more changes related to actually using .NET 8. We tried a few things in that PR to fix things, such as having Stryker use .NET 6, 7 and 8 to see if it was a configuration issue, or something we could side-step.

We have some .NET 8-specific changes we'd like to release in November once 8.0.0 is available, so if we can't get things working we're likely going to have to disable our mutation tests as otherwise it will be a blocker for that release as we have to use the .NET 8 SDK for those changes.

I happened to notice in the last hour that there was a new version of Stryker available, so I'm in the process of applying that update too in case that would resolve it.

Logs

Failed build logs here: logs

I'll attempt to produce a log file at a later point.

Expected behavior

The tests succeed the same way they do for us with the .NET 7 SDK.

Desktop (please complete the following information):

OS: Ubuntu 22.04
Type of project: .NET library
Framework Version: .NET 8.0.100 RC2 SDK
Stryker Version: 3.10.0

Additional context

None.

martincostello commented 10 months ago

Yep, still fails with 3.11.0 too.

rouke-broersma commented 10 months ago

Usually using preview versions of sdks is tricky because the roslyn package with support for the next tfm is released with the release of the next tfm. So to support a project trying to use preview versions we would have to update to preview versions of roslyn, which is undesirable for obvious reasons.

I'm not sure if that's the case here, but it's a possibility, we'll have to test if upgrading roslyn to a preview version would fix the issues.

martincostello commented 10 months ago

That makes sense. I wonder if the issue is caused by the default C# language version moving from 11 to 12?

I'll try pinning it to 11 to see if that fixes things. If it does, then we can just workaround the issue in the short term at the expense of not being able to use C# 12.

dupdob commented 10 months ago

could you adjust log levels for Stryker? Ideally, using --logToFile and providing us the resulting log? Stryker fails at analyzing the projects, clearly failing to capture dependencies. There is also a warning that net7.0 is requested while there is no configuration supporting it. SDK resolution is tricky and not handled by Stryker, and this is a crucial step. Changing C# version will not change a bit, I am afraid.

martincostello commented 10 months ago

Logs can be found in the mutation-report artifact for this GitHub Actions workflow run.

dupdob commented 10 months ago

Thanks. I was looking for an error message I though I added a few months ago, but I failed to see it. And I don't see it in Stryker source files. I guess it is in some yet to be merged PR. That being said, that error message would not have been super helpful, merely a confirmation of what is happening. Compilation error messages point toward Stryker/Builadalyzer failing to properly capture project references. We have no clear cause for this as of now.

martincostello commented 10 months ago

With some further experimentation I have found that I can get it to work on Windows (both locally and in GitHub Actions) by adding the following two settings:

{
+  "language-version": "Preview",
-  "target-framework": "net7.0"
+  "target-framework": "net8.0"
}

Stryker still fails on macOS and Linux with these changes.

Could it be related to #2585 somehow?

mu88 commented 9 months ago

Hello fellow Stryker developers 👋🏻

After upgrading my project to .NET 8 and using some of C# 12's latest language features (primary constructor), Stryker fails with:

[14:48:21 WRN] Safe Mode! Stryker will try to continue by rolling back all mutations in method. This should not happen, please report this as an issue on github with the previous error message.
[14:48:21 WRN] Stryker.NET encountered a compile error in /home/runner/work/RaspiFanController/RaspiFanController/RaspiFanController/Logic/RaspiTemperatureController.cs (at 4:39) with message: The feature 'primary constructors' is currently in Preview and *unsupported*. To use Preview features, use the 'preview' language version. (Source code: (ITemperatureProvider temperatureProvider,
                                        IFanController fanController,
                                        ITaskCancellationHelper taskCancellationHelper,
                                        ITaskHelper taskHelper,
                                        ILogger<RaspiTemperatureController> logger,
                                        IOptionsMonitor<AppSettings> settings))

I assume that is also a problem related to this issue? Or shall I better create a new issue?