Unhandled exception. System.InvalidOperationException, when using more than 1 test project.

[anderswagner]

Describe the bug When running stryker with a config using both Api.Test project and Library.Test project, i receive the following exception. Having to dig into the source code of stryker to figure out what could potentially make this error happen is not ideal.

Logs

   _____ _              _               _   _ ______ _______  
  / ____| |            | |             | \ | |  ____|__   __| 
 | (___ | |_ _ __ _   _| | _____ _ __  |  \| | |__     | |    
  \___ \| __| '__| | | | |/ / _ \ '__| | . ` |  __|    | |    
  ____) | |_| |  | |_| |   <  __/ |    | |\  | |____   | |    
 |_____/ \__|_|   \__, |_|\_\___|_| (_)|_| \_|______|  |_|    
                   __/ |                                      
                  |___/                                       

Version: 3.12.0

[10:49:02 INF] Analysis starting.
[10:49:11 INF] Found project ..\X\X.Library\X.Library.csproj to mutate.
[10:49:11 INF] Analysis complete.
[10:49:11 INF] Building test project ..\X\X.Library.Test\X.Library.Test.csproj (1/2)
[10:49:26 INF] Building test project ..\X\X.Api.Test\X.Api.Test.csproj (2/2)
[10:49:51 INF] Number of tests found: 60 for project ..\X\X.Library\X.Library.csproj. Initial test run started.
[10:50:00 INF] Time Elapsed 00:00:58.1671917
Unhandled exception. System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.ThrowHelper.ThrowMoreThanOneMatchException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Func`2 predicate, Boolean& found)
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at Stryker.Core.Initialisation.ProjectMutator.EnrichTestProjectsWithTestInfo(InitialTestRun initialTestRun, TestProjectsInfo testProjectsInfo) in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectMutator.cs:line 53
   at Stryker.Core.Initialisation.ProjectMutator.MutateProject(StrykerOptions options, MutationTestInput input, IReporter reporters) in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectMutator.cs:line 35
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters, ITestRunner runner)+MoveNext() in /_/src/Stryker.Core/Stryker.Core/Initialisation/ProjectOrchestrator.cs:line 68
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in /_/src/Stryker.Core/Stryker.Core/StrykerRunner.cs:line 61
   at Stryker.CLI.StrykerCli.RunStryker(IStrykerInputs inputs) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 93
   at Stryker.CLI.StrykerCli.<>c__DisplayClass10_0.<Run>b__0() in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 68
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass143_0.<OnExecute>b__0(CancellationToken _)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at Stryker.CLI.StrykerCli.Run(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 74
   at Stryker.CLI.Program.Main(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/Program.cs:line 14

Expected behavior A more specific error message than an unhandled exception, or otherwise a warning to let the stryker run keep moving forward

Desktop (please complete the following information):

• OS: Windows 11
• Project: API & Library
• Framework Version: .NET 6.0
• Stryker Version: dotnet-stryker 3.12

Additional context Stryker config file here, with the same naming:

{ "stryker-config": { "ignore-methods": [ "Arg.NotNull", "Error", "Warning", "Information", "GC.SuppressFinalize", "ConfigureAwait" ], "mutate": [ "!*/Humble.cs" ], "reporters": [ "html" ], "solution": "../X.sln", "project": "X.Library.csproj", "test-projects": [ "../X.Library.Test/X.Library.Test.csproj", "../X.Api.Test/X.Api.Test.csproj"] } }

And the working stryker config here

{ "stryker-config": { "ignore-methods": [ "Arg.NotNull", "Error", "Warning", "Information", "GC.SuppressFinalize", "ConfigureAwait" ], "mutate": [ "!*/Humble.cs" ], "reporters": [ "html" ], "solution": "../X.sln", "project": "X.Library.csproj", "test-projects": [ "../X.Library.Test/X.Library.Test.csproj"] } }

The same sort of setup has worked for other projects (Api.Test & Library.Test)

[dupdob]

the bug happens if you have the same test source files in two different test projects, which is indeed not supported as of today (because nobody though about this situation). This is a legitimate use case, but we need to understand the ramification to support this properly.

[VMelnalksnis]

I also was hit with this issue, @CptF1nn did you find a way to work around this until it's fixed?

[anderswagner]

@VMelnalksnis We neded up only using tests for one project against our library instead of multiple sadly