The test project binaries could not be found at D:---\ABCTests\Build\bin\Debug\MyLib.UnitTests.dll, exiting...

[AS25061988]

Describe the bug What is the fix for this issue.I am too facing the error The test project binaries could not be found at D:---\MutationTests\Build\bin\Debug\MyLib.UnitTests.dll, exiting...

Every time while running dotnet stryker --solution-path "D:..\Solution.sln" --project-file "D:..\Source.csproj" , it rebuilds the test project and at the same time expects the binary also in the ..bin\debug path.

Logs The test project binaries could not be found at D:---\ABCTests\Build\bin\Debug\MyLib.UnitTests.dll, exiting...

Expected behavior A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

• OS: Windows10
• Type of project [C#, .NET Framework 4.7.2]
• Framework Version [4.7.2]
• Stryker Version [0.19.0 (beta)]

Additional context Add any other context about the problem here.

[rouke-broersma]

Could you show us your project structure?

[psfinaki]

Hey there! We are ramping up new repo on Stryker and getting this exact issue for a bunch of projects, i.e. GeneralStrykerException in InitializeVsTestConsole.

I was thinking what to do, the full log file contains too much NDA 🤦

Let's start with -V debug and then I'll answer your questions on whatever extra info you need :D

[19:41:10 INF] Logging enabled at level Info
Version: 1.5.0

[19:41:11 DBG] Stryker started with options: {"MsBuildPath": null, "DevMode": false, "BasePath": "REPO\\src\\TESTPROJECT", "OutputPath": "REPO\\src\\TESTPROJECT\\StrykerOutput\\2022-03-29.19-41-10", "ReportFileName": "mutation-report", "SolutionPath": "..\\SOLUTION.sln", "TargetFramework": null, "LogOptions": {"LogToFile": false, "LogLevel": "Debug", "$type": "LogOptions"}, "MutationLevel": "Standard", "Thresholds": {"High": 80, "Low": 60, "Break": 0, "$type": "Thresholds"}, "AdditionalTimeout": 5000, "LanguageVersion": "Default", "Concurrency": 4, "ProjectUnderTestName": "", "TestProjects": [], "TestCaseFilter": "", "WithBaseline": false, "Reporters": ["Progress", "Html"], "BaselineProvider": "Disk", "AzureFileStorageUrl": "", "AzureFileStorageSas": "", "DashboardUrl": "https://dashboard.stryker-mutator.io", "DashboardApiKey": null, "Since": false, "SinceTarget": "master", "DiffIgnoreChanges": [], "FallbackVersion": "master", "ModuleName": "", "ReportTypeToOpen": null, "Mutate": [{"Glob": {"Tokens": [{"TrailingPathSeparator": {"Value": "\\", "$type": "PathSeparatorToken"}, "LeadingPathSeparator": null, "$type": "WildcardDirectoryToken"}, {"$type": "WildcardToken"}], "$type": "Glob"}, "IsExclude": false, "TextSpans": [{"Start": 0, "End": 2147483647, "Length": 2147483647, "IsEmpty": false, "$type": "TextSpan"}], "$type": "FilePattern"}], "IgnoredMethods": [], "ExcludedMutations": [], "ExcludedLinqExpressions": [], "OptimizationMode": "CoverageBasedTest", "ProjectName": "", "ProjectVersion": "", "$type": "StrykerOptions"}
[19:41:11 INF] Identifying project to mutate.
[19:41:11 DBG] Using TESTPROJECT as test project
[19:41:11 DBG] Analyzing solution file ..\SOLUTION.sln
[19:41:11 DBG] Analyzing project file REPO\src\TESTPROJECT\TESTPROJECT.csproj
[19:41:17 DBG] Using REPO\src\PROJECT\PROJECT.csproj as project under test
[19:41:17 INF] The project REPO\src\PROJECT\PROJECT.csproj will be mutated.
[19:41:17 DBG] Analyzing solution file ..\SOLUTION.sln
[19:41:17 DBG] Analyzing project file REPO\src\PROJECT\PROJECT.csproj
[19:41:22 DBG] Skipping auto-generated code file: REPO\obj\PROJECT\net472\.NETFramework,Version=v4.7.2.AssemblyAttributes.cs
[19:41:22 DBG] Skipping auto-generated code file: REPO\obj\PROJECT\net472\PROJECT.AssemblyInfo.cs
[19:41:22 INF] Analysis complete.
[19:41:22 INF] Building test project REPO\src\TESTPROJECT\TESTPROJECT.csproj (1/1)
[19:41:22 DBG] Started initial build using msbuild.exe
[19:41:22 DBG] Using vswhere.exe to locate msbuild
[19:41:23 DBG] Msbuild executable path found at C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\MSBuild.exe
[19:41:53 DBG] Initial build successful
[19:41:53 DBG] Using vstest from nuget package folders
[19:41:53 DBG] Using vstest.console: C:\Users\psemkin\.nuget\packages\microsoft.testplatform.portable\17.1.0\tools\net451\vstest.console.exe for OS WINDOWS
[19:41:53 INF] Time Elapsed 00:00:42.1690439
Unhandled exception.

The test project binaries could not be found at REPO\bin\TESTPROJECT\net472\TESTPROJECT.dll, exiting...

[psfinaki]

So the testBinariesPaths are "REPO\bin\TESTPROJECT\\net472\\TESTPROJECT.dll". Indeed they don't exist:

What does exist is REPO\bin\TESTPROJECT\Debug\net472\TESTPROJECT.dll :D Once I hardcoded this, things started to move.

So it should be an easy fix somewhere but do you have a guess how this could slip for such a long while?

[rouke-broersma]

We get this path from buildalyzer, for some reason for some projects buildalyzer returns the wrong binary path.

[psfinaki]

Okay so I confirm that, the issue is further down the line. There is some MSBuild voodoo in the project for output paths for the binaries, so it's either Buildalyzer not handling this properly or MSBuild itself. Once I hacked things a bit, Stryker started working correctly.

So we'll either keep investigating this or we'll remove the MSBuild voodoo - either way for now I revoke my complaint so feel free to close this again :)

[rouke-broersma]

It is still a problem we would like to solve, because you are not the first to hit this. If you could give us some insights into this msbuild voodoo that changes the output paths perhaps we could figure out a way to find this in buildalyzer or stryker.

[psfinaki]

Oh, that for sure, here I can tell the whole story :D The .sln contains a bunch of normal C# projects (.csproj) and a few weirder things like .ccproj (doesn't really matter what this is). Directory.build.props for the solution contained this prop:

  <PropertyGroup>
    <OutputPath>$(MSBuildThisFileDirectory)..\bin\$(MSBuildProjectName)\$(Configuration)\</OutputPath>
  </PropertyGroup>

So first I just removed this thing from the props file. The solution compiled but when executing Stryker, it threw a bunch of build errors for this .ccproj and another project I was not interested in. So I added the above prop to those two project files.

After that, the solution kept compiling and Stryker started working for the project I cared about. So yeah, it's one of those cases when I don't know what I'm doing and it somehow works, so we're trying to find people who understand what's going on here.

[psfinaki]

But well, as you see, the effect of this is Stryker mislooking the binaries for one level so a very local fix would be to search recursively in a broader space (like, in the whole "bin" folder).

[bdukes]

I'm seeing this issue every time I try to use Stryker, I've pushed an example repository. I didn't try moving the projects around, but I removed everything that wasn't required for the example and am still seeing the issue. Hope it helps!

[rouke-broersma]

I think this is related to #2077

From analysis for that issue it seemed that we simply don't have the required information to determine that there is an intermediate folder (in your case Any CPU) in the output folder. For some reason msbuild knows about this, but we cannot find this information anywhere.

[bdukes]

I do not see this issue running in WSL, so it looks like it may be a Windows-specific issue

[bdukes]

Thanks. Just for clarity, I don't have an Any CPU folder, but Stryker is looking for one.

[rouke-broersma]

That's interesting, that's the reverse problem. So in this case we are detecting an intermediate folder, probably from the sln where the Any CPU profiles are specified. Note that solution runs still have some issues which we hope to fix soon. You might have better results running against a single source project.

[rouke-broersma]

Since we have not had a report of this issue in a while and since we've recently had some improvements in this area I will close this for now.