stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.78k stars 188 forks source link

Stryker hanging when running on stryker sln #2641

Closed richardwerkman closed 7 months ago

richardwerkman commented 1 year ago

Describe the bug When running dotnet stryker on stryker itself we get an unexpected exception:

Version: 3.10.0

[10:46:16 INF] Analysis starting.
[10:46:16 INF] Identifying projects to mutate in C:\Dev\Repos\stryker-net\src\Stryker.sln. This can take a while.
[10:46:17 WRN] Analysis of project Stryker.RegexMutators\Stryker.RegexMutators\Stryker.RegexMutators.csproj failed.
[10:46:17 WRN] Analysis of project Stryker.RegexMutators\Stryker.RegexMutators.UnitTest\Stryker.RegexMutators.UnitTest.csproj failed.
[10:46:17 WRN] Analysis of project Stryker.CLI\Stryker.CLI\Stryker.CLI.csproj failed.
[10:46:17 WRN] Analysis of project Stryker.CLI\Stryker.CLI.UnitTest\Stryker.CLI.UnitTest.csproj failed.
[10:46:17 WRN] Analysis of project Stryker.Core\Stryker.Core\Stryker.Core.csproj failed.
[10:46:17 WRN] Analysis of project Stryker.Core\Stryker.Core.UnitTest\Stryker.Core.UnitTest.csproj failed.
[10:46:29 INF] Found 1 source projects
[10:46:29 INF] Found 0 test projects
[10:46:29 WRN] Project Stryker.DataCollector\Stryker.DataCollector\Stryker.DataCollector.csproj will not be mutated because Stryker did not find a test project for it.
[10:46:29 INF] Analysis complete.
[10:46:29 INF] Building solution Stryker.sln
[10:47:11 INF] Time Elapsed 00:00:54.9610941
Unhandled exception. System.InvalidOperationException: Sequence contains no elements
   at System.Linq.ThrowHelper.ThrowNoElementsException()
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters, ITestRunner runner)+MoveNext() in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectOrchestrator.cs:line 57
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in /_/src/Stryker.Core/Stryker.Core/StrykerRunner.cs:line 61
   at Stryker.CLI.StrykerCli.RunStryker(IStrykerInputs inputs) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 93
   at Stryker.CLI.StrykerCli.<>c__DisplayClass10_0.<Run>b__0() in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 68
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass143_0.<OnExecute>b__0(CancellationToken _)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at Stryker.CLI.StrykerCli.Run(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 74
   at Stryker.CLI.Program.Main(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/Program.cs:line 14

Reproduction Run stryker in the src/ dir of the stryker repo.

Expected behavior Stryker runs on the solution file and succeeds.

Desktop (please complete the following information):

Additional context I've investigated buildalyzer to be the source of the issue. Somehow buildalyzer fails to build all projects except for DataCollector. However I haven't figured out WHY buildalyzer is failing to build these projects.

dupdob commented 1 year ago

I just tried and I can't even pass through the analysis phase: I get a missing DLL exception with Buildalizer. Weird... Update: enumerating Buildalyzer results lead to file not found exception for nuget.Framework 6.0, while the project (and buildalyzer) refers to 6.7... crap Fuslogvw shows that the failed attempts from MsBuild. I suspect some global prop file here....

Re Update: I am facing this problem: https://github.com/daveaglick/Buildalyzer/issues/206

dupdob commented 1 year ago

This works ok if I use Stryker against a copy of the project. In short, you cannot use Buildalyzer against the currently running project (see https://github.com/daveaglick/Buildalyzer/issues/206) Otherwise weird shit happens :-)

rouke-broersma commented 1 year ago

@dupdob we have this problem when running on a copy 😅

dupdob commented 1 year ago

Sorry about that. No problem here, using master/head With VS 2022 but not sure this is a factor

richardwerkman commented 1 year ago

To be clear, I'm not only having this issue while debugging stryker but also using the actual installed tool (latest version). Just running dotnet stryker in the src directory.

richardwerkman commented 1 year ago

@dupdob and you're running on windows? Maybe the OS has influence on this

dupdob commented 1 year ago

yes, windows update: it also works with the distribution. Do you have the same issue with master/Head ? maybe it is due to some local change(s)

richardwerkman commented 1 year ago

To test this I cloned stryker fresh on another machine and ran stryker there. It worked! So indeed there seems to be something wrong with my local machine. I've tested there on 2 seperate stryker repos and both gave the error. I'll investige further if this is thanks to local changes like you suggest, or if this is because of the state of my machine

richardwerkman commented 1 year ago

After some further investigation (and asking others to try running stryker on stryker) these are my findings:

dupdob commented 1 year ago

@richardwerkman: nope, Windows 11 here

dupdob commented 1 year ago

I can check again if you want, but it works on my machine with a copy of the project

richardwerkman commented 1 year ago

In that case it could also be the dotnet version. I'm on the latest (7.0.401)

dupdob commented 1 year ago

but Stryker is built against Net 6

dupdob commented 1 year ago

here are the runtimes I have


 Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-preview.5.23302.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-preview.7.23375.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0-preview.5.23280.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0-preview.7.23375.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0-preview.5.23302.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0-preview.7.23376.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]```
dupdob commented 7 months ago

I close this issue as it looks like a transient/setup issue. Feel free to reopen it with an update if the issue is till present