Cleanup msbuild locate

[richardwerkman]

We currently locate msbuild by calling vswhere inside a visual studio installation dir. On our buildserver vswhere could not be found, so as a workaround we placed a hardcoded lookup for the msbuild location on the build server.

https://github.com/stryker-mutator/stryker-net/blob/ba5e4a281c201b2e36576f7693abfab94fc35a89/src/Stryker.Core/Stryker.Core/ToolHelpers/MsBuildHelper.cs#L33

This now results in strange behaviour, as vs2017 enterprise is now always picked first. Even when vs2019 is installed on a system. We should find a way to locate msbuild without a hardcoded lookup on both dev machines and build servers.

[robertlyson]

@Mobrockers did some research on this topic https://github.com/stryker-mutator/stryker-net/issues/369, don't know what are his thoughts. Also FAKE and cake are dealing with it, maybe that will put some light on the problem.

[richardwerkman]

@robertlyson Do you think we can use cake to resolve msbuild? I'm not familliar with cake myself so I'm not sure if that is correct usage of cake.

[robertlyson]

I don't think we want to have a dependency on cake in stryker-net, that would be cumbersome. Microsoft.Build.Utilities.Core seems like a much more lighter solution.

[CrispyDrone]

Hello, I'm trying to get stryker to work for a project at work but experiencing issues with stryker not being able to locate MSBuild using vswhere, and instead defaulting to the version that's installed in the framework location. However when it then executes the restore command it passes an invalid option to the MsBuildVersion flag.

Here is the trace level log (Replaced "sensitive" information):

[13:15:30 DBG] Stryker started with options: {"BasePath":"E:\\MyCompany\\MyCompany.MyProject\\MyCompany.MyProject.Domain.Tests","SolutionPath":"E:\\MyCompany\\MyCompany.MyProject\\MyCompany.MyProject.Domain.Tests\\../MyCompany.MyProject.sln","OutputPath":"E:\\MyCompany\\MyCompany.MyProject\\MyCompany.MyProject.Domain.Tests\\StrykerOutput\\2019-06-24.13-15-30","Reporters":["Progress","ClearText"],"LogOptions":{"LogToFile":false,"OutputPath":"E:\\MyCompany\\MyCompany.MyProject\\MyCompany.MyProject.Domain.Tests\\StrykerOutput\\2019-06-24.13-15-30","LogLevel":"Verbose"},"DevMode":false,"ProjectUnderTestNameFilter":"../MyCompany.MyProject.Domain/MyCompany.MyProject.Domain.csproj","AdditionalTimeoutMS":30000,"ExcludedMutations":[],"ConcurrentTestrunners":4,"Thresholds":{"High":80,"Low":60,"Break":0},"TestRunner":"DotnetTest","FilesToExclude":[]}
[13:15:30 VRB] Scanned the current directory for *.csproj files: found E:\MyCompany\MyCompany.MyProject\MyCompany.MyProject.Domain.Tests\MyCompany.MyProject.Domain.Tests.csproj
[13:15:30 INF] Using E:\MyCompany\MyCompany.MyProject\MyCompany.MyProject.Domain.Tests\MyCompany.MyProject.Domain.Tests.csproj as project file
[13:15:30 DBG] Analyzing solution file E:\MyCompany\MyCompany.MyProject\MyCompany.MyProject.Domain.Tests\../MyCompany.MyProject.sln
[13:15:30 DBG] Analyzing project file E:\MyCompany\MyCompany.MyProject\MyCompany.MyProject.Domain.Tests\MyCompany.MyProject.Domain.Tests.csproj
[13:15:31 DBG] Project analyzer result not successful, restoring packages
[13:15:31 INF] Restoring nuget packages using nuget.exe
[13:15:31 DBG] Using vswhere.exe to locate msbuild
[13:15:31 DBG] Unable to find msbuild using vswhere, using fallback locations
[13:15:31 DBG] Auto detected msbuild version 4.8.3752.0 at: C:\Windows\Microsoft.Net\Framework64\v4.0.30319\MSBuild.exe
[13:15:31 DBG] Restoring packages using command: C:\Users\myuser\bin\nuget.exe restore "E:\MyCompany\MyCompany.MyProject\MyCompany.MyProject.sln" -MsBuildVersion "4.8.3752.0"
[13:15:31 INF] Time Elapsed 00:00:01.6419808
Stryker.NET failed to mutate your project. For more information see the logs below:

Nuget.exe failed to restore packages for your solution. Please review your nuget setup.

Cannot find the specified version of msbuild: '4.8.3752.0'

[rouke-broersma]

@CrispyDrone We get the msbuild version by executing MSBuild.exe /version. For some reason your msbuild returns an incorrect version? We then pass this version number to nuget.exe to make sure nuget uses the same msbuild version for the restore as the compilation will use. Do you have any suggestion as what we can improve here?

[richardwerkman]

Hi @CrispyDrone could you please run nuget.exe restore without the extra commands by hand? I'm curious what msbuild version it will use. We pass the specific version to nuget.exe as msbuild requires that nuget used exactly the same version to restore, to build the project succesful.

[CrispyDrone]

Thank you for the quick responses. I didn't get round to giving it another try yesterday, sorry for that.

I just executed the nuget restore -SolutionDirectory ../ command. It only returns the following (after manually deleting the packages folder), nothing about which version of msbuild it's using. But I might have misunderstood what you meant.

Restoring NuGet package FluentValidation.8.0.100.
Restoring NuGet package Castle.Core.4.3.1.
Restoring NuGet package System.Collections.Concurrent.4.3.0.
Restoring NuGet package FluentAssertions.5.2.0.
Restoring NuGet package Moq.4.7.145.
Restoring NuGet package EntityFramework.6.1.3.
Restoring NuGet package System.ComponentModel.Annotations.4.4.1.
Restoring NuGet package StructureMap.4.6.1.
Adding package 'Castle.Core.4.3.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'FluentAssertions.5.2.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'System.Collections.Concurrent.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'StructureMap.4.6.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'EntityFramework.6.1.3' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'System.ComponentModel.Annotations.4.4.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'FluentValidation.8.0.100' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'Moq.4.7.145' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'Moq.4.7.145' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package System.ComponentModel.Primitives.4.3.0.
Added package 'StructureMap.4.6.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package System.Reflection.Emit.Lightweight.4.3.0.
Adding package 'System.ComponentModel.Primitives.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'FluentValidation.8.0.100' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package System.Runtime.CompilerServices.Unsafe.4.5.0.
Adding package 'System.Reflection.Emit.Lightweight.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Adding package 'System.Runtime.CompilerServices.Unsafe.4.5.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'Castle.Core.4.3.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package System.Threading.Tasks.Extensions.4.5.1.
Adding package 'System.Threading.Tasks.Extensions.4.5.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'FluentAssertions.5.2.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package System.ValueTuple.4.5.0.
Adding package 'System.ValueTuple.4.5.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.Runtime.CompilerServices.Unsafe.4.5.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Restoring NuGet package VS.QualityTools.UnitTestFramework.15.0.27323.2.
Adding package 'VS.QualityTools.UnitTestFramework.15.0.27323.2' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.ComponentModel.Primitives.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.Threading.Tasks.Extensions.4.5.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.Collections.Concurrent.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'VS.QualityTools.UnitTestFramework.15.0.27323.2' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.ComponentModel.Annotations.4.4.1' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.Reflection.Emit.Lightweight.4.3.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'System.ValueTuple.4.5.0' to folder 'E:\MyCompany\MyCompany.MyProject\packages'
Added package 'EntityFramework.6.1.3' to folder 'E:\MyCompany\MyCompany.MyProject\packages'

NuGet Config files used:
    C:\Users\myuser\AppData\Roaming\NuGet\NuGet.Config
    C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

Feeds used:
    C:\Users\myuser\.nuget\packages\
    https://api.nuget.org/v3/index.json
    http://tfs.MyCompany.com/tfs/DefaultCollection/_packaging/MyCompany.Framework/nuget/v3/index.json
    C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

Installed:
    14 package(s) to packages.config projects

When I look at the command line reference for nuget restore ( https://docs.microsoft.com/en-us/nuget/tools/cli-ref-restore ), it just seems to indicate that it does not support this exact version:

(3.2+) Specifies the version of MSBuild to be used with this command. Supported values are 4, 12, 14, 15.1, 15.3, 15.4, 15.5, 15.6, 15.7, 15.8, 15.9. By default the MSBuild in your path is picked, otherwise it defaults to the highest installed version of MSBuild.

I don't have any suggestions on how to improve the flow for fetching the MSBuildPath/Version and subsequently using it. I'm mainly wondering what I can do to make vswhere be able to find my installed version in my path (since I actually do have MSBuild in my path), or to have stryker work with the default installed framework version :)

In fact here is the output for vswhere --nologo -latest:

instanceId: d03718e2
installDate: 07/02/2018 10:34:36
installationName: VisualStudio/15.9.11+28307.586
installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional
installationVersion: 15.9.28307.586
productId: Microsoft.VisualStudio.Product.Professional
productPath: C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe
isPrerelease: 0
displayName: Visual Studio Professional 2017
description: Professional developer tools and services for small teams
channelId: VisualStudio.15.Release
channelPath: C:\Users\IPK\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\4CB340F5\catalog.json
channelUri: https://aka.ms/vs/15/release/channel
enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service
releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660692#15.9.11
thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660708
updateDate: 2019-04-24T09:35:35.1906036Z
catalog_buildBranch: d15.9
catalog_buildVersion: 15.9.28307.586
catalog_id: VisualStudio/15.9.11+28307.586
catalog_localBuild: build-lab
catalog_manifestName: VisualStudio
catalog_manifestType: installer
catalog_productDisplayVersion: 15.9.11
catalog_productLine: Dev15
catalog_productLineVersion: 2017
catalog_productMilestone: RTW
catalog_productMilestoneIsPreRelease: False
catalog_productName: Visual Studio
catalog_productPatchVersion: 11
catalog_productPreReleaseMilestoneSuffix: 1.0
catalog_productRelease: RTW
catalog_productSemanticVersion: 15.9.11+28307.586
catalog_requiredEngineVersion: 1.18.1049.33485
properties_campaignId:
properties_canceled: 0
properties_channelManifestId: VisualStudio.15.Release/15.9.11+28307.586
properties_nickname:
properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installershell.exe

[rouke-broersma]

Buildalyzer: https://github.com/daveaglick/Buildalyzer

Is pretty consistent in being able to locate the correct msbuild versions. We should take a look there to see how we can properly do this in stryker.

[dupdob]

hi, I am trying to clean up old issues, and this is the oldest. what should be the validation test for this ? alternatively, how can I reproduce the issue?

Looking at the code, I see several ways to improve on the search code and making it more generic, but without a validation test, I can't be sure it has any interest.

[dupdob]

hello I have a fork relying on buildalyzer to locate msbuild. It works on every project I have, but it looks like it favors de netcore variant (msbuild.dll) over the executable. For me, this change makes complete sense as Stryker cannot work if buildalyzer fails do find MsBuild. Note that providing a custom path is still supported (and now supports providing a path to the assembly instead of the executable)

[richardwerkman]

That sounds like it fixes this issue! It does however increase our reliance on buildalyzer. This is a risk, if buildalyzer ever would stop being supported. But for now, I like this as a solution. Could you provide a PR for this?

[dupdob]

Regarding reliance to Buildalyzer, it is easy to duplicate its logic if need arises. Actually, that was my first intention, but I realized this was a waste of effort. The codebase needs some adjustment before a PR to be ready: MsBuild discovery happens within two different parts (nuget and build), many tests need to be adjusted.... I needed for positive feedback before engaging in the grunt work