Stryker doesn't find test project DLLs for x64 only projects

[matisskeiris]

Describe the bug When running Stryker for x64 only project it fails to find Test project DLLs.

Logs log-20220608.txt

Expected behavior It should find DLLs built by MSBuild

Desktop (please complete the following information):

• OS: Windows
• Type of project: framework
• Framework Version: net472
• Stryker Version: 2.0.0

Additional context Example solution reproducing the issue: https://github.com/matisskeiris/stryker-x64

[rouke-broersma]

The provided log file seems to end at

Initial build successful (61599426) 2022-06-08T20:56:09.5302186+03:00 [INF] Time Elapsed 00:00:06.1394719 (3923d839)

Is this my mobile viewer messing up, was the rest omitted from the provided log or was the actual error simply never logged to the log file?

I will run the provided reproduction tomorrow but could you confirm what exact error you received?

[matisskeiris]

It just fails with error message

Failed to locate project binary at C:/repos/stryker-x64/TestingStrykerIssues/bin/Debug/net472/win-x64/TestingStrykerIssues.dll

It doesn't append x64 to the bin location, as the real location of binary is here C:/repos/stryker-x64/TestingStrykerIssues/bin/x64/Debug/net472/win-x64/TestingStrykerIssues.dll

[matisskeiris]

One way to fix this problem is to add in project Output Directory property, but that seems like a hacky solution because the project and solution builds and runs tests as expected through VS and dotnet cli

[rouke-broersma]

This sounds like an issue that has been reported a couple times before. We use a tool called buildalyzer to gather information about the project from msbuild, and for some reason in some cases (mostly on netframework) the output directory we get from msbuild is incorrect. Unfortunately I don't have enough knowledge about msbuild to know why the output directory is incorrect and what vstest uses to find the correct binary location to execute the tests in these cases.

Would you perhaps have some information on how we can find the correct output directory?

Edit: I looked through every single msbuild property for the example project, there are exactly 0 properties with the value 'bin/x64/Debug', every single property that has anything to do with output is set to '/bin/Debug'. The only properties specifying x64 are the PlatformTarget but I did not yet find any indication that this should be treated as something that should be mashed in the middle of the output paths.

Edit: From here: https://stackoverflow.com/questions/72222774/msbuild-on-net-5-core-projects-produces-different-bin-x64-and-bin-debug-folder-s I gather that the reason is indeed solely that there is no AnyCPU PlatformTarget, except it specifies that the targetdir/outdir/outputpath should contain the PlatformTarget in the path already, so for some reason it is malformed coming out of buildalyzer or out of msbuild.

[rouke-broersma]

Since we have not had a report of this issue in a while and since we've recently had some improvements in this area I will close this for now.

[tisonv]

Hello

Problem still exists I have a configuration called Dev with which I run my tests. The project is only available as x64

The following all fails building :

 dotnet stryker --configuration Dev
 dotnet stryker --configuration "Dev x64"
 dotnet stryker --configuration "x64 Dev"

As a test, I added the "Any CPU" configuration and dotnet stryker --configuration Dev then builds successfully

[dupdob]

What is the failure/error message ? x64 architecture target do work on our integration test, so this is not a trivial error

[tisonv]

Here you are :

dotnet stryker --configuration CI --project "Infrastructure"

<PropertyGroup>
  <TargetFramework>net8.0-windows10.0.22621.0</TargetFramework>
  <ImplicitUsings>enable</ImplicitUsings>
  <Nullable>enable</Nullable>

  <AssemblyName>_Omitted_</AssemblyName>
  <RootNamespace>_Omitted_</RootNamespace>
  <Configurations>[...],CI</Configurations>
  <Platforms>x64</Platforms>

  <EnforceCodeStyleInBuild>True</EnforceCodeStyleInBuild>
  <EnableNETAnalyzers>True</EnableNETAnalyzers>
  <AnalysisLevel>latest-all</AnalysisLevel>

  <SupportedOSPlatformVersion>10.0.22621.0</SupportedOSPlatformVersion> 
</PropertyGroup>

[dupdob]

Thanks, I have further questions:

1. Is bin/CI/net8...../ the expected target folder?
2. Do the test assemblies exist within this folder? if not, are they elsewhere?
3. What happens when you build the project using donet build xxxx -c CI

[tisonv]

1. No. It should be bin\x64\CI\net8.0-windows10.0.22621.0
2. In bin\CI\net8.0-windows10.0.22621.0, there's only testhost.exeand its dependencies Yes they are in bin\x64\CI\net8.0-windows10.0.22621.0
3. No bin\CI\net8.0-windows10.0.22621.0is created In bin\x64\CI\net8.0-windows10.0.22621.0, the test project and its dependencies are built

[dupdob]

Thanks. Short description of this issue is that the x64 part of the folder is somehow lost for Stryker. So this is a folder resolution issue. I can work this one, assuming I am able to reproduce it.

[dupdob]

Humm, I though this was simply a problem of forwarding the configuration to Buildalyzer, but we do it properly. I have tested on an AnyCpu project, adding a 'Debug x64' configuration and everything works properly... not sure where it comes from...

[tisonv]

The build in itself seems correct and end up in the correct folder Only the test tools and the execution are wrong. I don't know where the tools should be but only the path they got to the tests is wrong I'd say

bin\CI\net8.0-windows10.0.22621.0 is not more wrong than any other path imo.

[dupdob]

internally, we use Buildalyzer which logs what happens during the initial build to capture all needed information, including target directory. Stryker does not try to rebuild the path in any way, it uses the one Buildalyzer discovers. That is why it is hard to explain why it fails. It may be due to some msbuild targets being specific to this platform version (net8.0-windows10.0.22621.0) that somehow confuses buildalyzer.

To be very precise buildalyzer actually uses a fake build to discover all details, but this is not relevant here.

[dupdob]

The build in itself seems correct and end up in the correct folder Only the test tools and the execution are wrong. I don't know where the tools should be but only the path they got to the tests is wrong I'd say

bin\CI\net8.0-windows10.0.22621.0 is not more wrong than any other path imo.

Thanks. The build ends up in the correct folder because Stryker does a normal build (for test projects). The problem is to understand why Stryker/buildalyzer got bin\CI\net8.0-windows10.0.22621.0 instead of bin\x64\CI\net8.0-windows10.0.22621.0 from the TargetDir MsBuild property.

Any chance you could try extracting a minimal reproducing project/solution? stripping everything from this project for example. I understand this requires significant time, but it could be helpful.

On the other hand, the minimal project provided in the issue description worked perfectly on my machine the last time I tried it.

[tisonv]

Maybe simplier if you could you make a short snippet of how you call Buildalyzer ? I could run it against my projects

[dupdob]

Good idea. I would then suggest using Buildalyzer project instead. It would probably be easier and if you find the cause of the issue you could devise a fix along the way. In any case, it will provide you the basic logic on how to call BuildAlyzer. Here is the code for the main integration test that is used against a lot of different projects. https://github.com/phmonte/Buildalyzer/blob/main/tests/Buildalyzer.Tests/Integration/SimpleProjectsFixture.cs

[tisonv]

I didn't have much time to investigate this week. The problem originates from Buildalyzer and not Striker. I managed to get a bare Buildaliyzer working

AnalyzerManager manager = new AnalyzerManager();
IProjectAnalyzer analyzerResult = manager.GetProject(@"F:\XXXXX\Api.csproj");
analyzerResult.SetGlobalProperty("configuration", "CI");
//analyzerResult.SetGlobalProperty("platform", "x64");
//analyzerResult.SetGlobalProperty("runtime", "win-x64");
var results = analyzerResult.Build();

With AnyCPU on the project, it works, results.OverallSuccess = true. Every output files are where they need to be.

Without it, it fails. Forcing x64or win-x64has no effect.

I have not been able to find an error message in the AnalyzerResults though.

I copied / pasted the AnalyzerResults .CompilerCommand but it won't work from the command line as too long ?!