BuildManager.Build unable to build a CSPROJ path

[mwpowellhtx]

I get the log below when I try to invoke the BuildManager.Build for a .csproj Project.

The same invocation for the containing .sln Solution works just fine. The output I receive is the usual set of unresolved assemblies:

My log from the build using Detailed verbosity threshold:

Build started: 2018-08-20T15:01:02.1295538-04:00: Build started.
Project started: 2018-08-20T15:01:03.3166217-04:00: Project "AssyVersion_NetStandard.csproj" (Rebuild target(s)): Rebuild
Error: 2018-08-20T15:01:04.1486693-04:00 The "ResolveAssemblyReference" task failed unexpectedly.
System.InvalidOperationException: Failed to compare two elements in the array. ---> System.TypeInitializationException: The type initializer for 'Microsoft.Build.Shared.FileMatcher' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'System.Collections.Immutable, Version=1.2.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Microsoft.Build.Shared.FileMatcher..cctor()
   --- End of inner exception stack trace ---
   at Microsoft.Build.Tasks.TaskItemSpecFilenameComparer.Compare(ITaskItem x, ITaskItem y)
   at System.Array.SorterObjectArray.SwapIfGreaterWithItems(Int32 a, Int32 b)
   at System.Array.SorterObjectArray.PickPivotAndPartition(Int32 lo, Int32 hi)
   at System.Array.SorterObjectArray.IntroSort(Int32 lo, Int32 hi, Int32 depthLimit)
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)
   --- End of inner exception stack trace ---
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)
   at System.Array.Sort(Array keys, Array items, Int32 index, Int32 length, IComparer comparer)
   at System.Array.Sort(Array array, IComparer comparer)
   at Microsoft.Build.Tasks.ReferenceTable.GetReferenceItems(ITaskItem[]& primaryFiles, ITaskItem[]& dependencyFiles, ITaskItem[]& relatedFiles, ITaskItem[]& satelliteFiles, ITaskItem[]& serializationAssemblyFiles, ITaskItem[]& scatterFiles, ITaskItem[]& copyLocalFiles)
   at Microsoft.Build.Tasks.ResolveAssemblyReference.Execute(FileExists fileExists, DirectoryExists directoryExists, GetDirectories getDirectories, GetAssemblyName getAssemblyName, GetAssemblyMetadata getAssemblyMetadata, GetRegistrySubKeyNames getRegistrySubKeyNames, GetRegistrySubKeyDefaultValue getRegistrySubKeyDefaultValue, GetLastWriteTime getLastWriteTime, GetAssemblyRuntimeVersion getRuntimeVersion, OpenBaseKey openBaseKey, GetAssemblyPathInGac getAssemblyPathInGac, IsWinMDFile isWinMDFile, ReadMachineTypeFromPEHeader readMachineTypeFromPEHeader)
   at Microsoft.Build.Tasks.ResolveAssemblyReference.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() in /_/src/Build/BackEnd/TaskExecutionHost/TaskExecutionHost.cs:line 631
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext() in /_/src/Build/BackEnd/Components/RequestBuilder/TaskBuilder.cs:line 787
Build finished: 2018-08-20T15:01:04.1956719-04:00: Build FAILED.

Incidentally, I am preceding the call to Build with a programmatic NuGet Restore, which itself seemed to be necessary, and also appears to be working, no errors, exit code 0, whether I specify the Solution file or the Project file.

Background bits, I've identified the proper 15.0 Toolset, although perhaps I need to be on the latest 15.8? I'm not sure. Other than that, I am setting all the customary environment variables, etc, required to invoke MSBuild via Microsoft.Build.Execution.BuildManager.DefaultBuildManager, or so I believe.

Let me know if you need any other notes, I will try to furnish them.

Thanks!

[rainersigwald]

It sounds like you're running from a copy of Microsoft.Build.* assemblies that are up to date, but don't have an up-to-date copy of System.Collections.Immutable available for resolution.

Are you deploying System.Collections.Immutable.dll with your application? What version?

Are you using Microsoft.Build.Locator? Mentioning that you've set environment variables makes me think you're not.

[mwpowellhtx]

@rainersigwald Sounds accurate. I don't think so, re: Locator. I'll try to enumerate my steps, although it's not a straight shot, per se, as I've got things more or less wired evently if you will for purposes of my unit test services:

1. Starting from a using statement ProjectCollection instance pc.
2. Using pc I predicate the Toolsets based on ts.ToolsVersion=="15.0".
3. From there I derive the Installed Directory Path using ts.ToolsPath
4. I set a GlobalProperties dictionary to { {"Configuration", "Debug"}, {"Platform", "Any CPU"} }, although ostensibly I might like to inject that via my test case arguments eventually.
5. I am wiring in BuildParameters including my TestLogger (which provided the output above), i.e. new BuildParameters(pc) {Loggers = e.Loggers.ToArray()} where pc is the ProjectCollection.
6. Request data looks like this: new BuildRequestData(e.ProjectOrSolutionFullPath, e.GlobalProperties, ts.ToolsVersion, e.TargetsToBuild.ToArray(), null), where literally ProjectOrSolutionFullPath is the path to my Solution or Project.
7. Finally, I am capturing a var result = DefaultBuildManager.Build(parameters, requestData), for which I am given the expected result, namely Success.

The outer context has basically two steps:

1. Invoke the NuGet Restore on the Project or Solution, both of which seem to be working.
2. I Run the actual build steps, enumerated above.

The environment variables I am setting at the moment are:

SetEnvironmentVariable("VSINSTALLDIR", e.InstallDirectoryName);
SetEnvironmentVariable("MSBUILD_EXE_PATH", e.Toolset.ToolsPath);
SetEnvironmentVariable("VisualStudioVersion", e.Toolset.ToolsVersion);

Although, I'm not sure these are strictly necessary, at least they did not seem to be, but I could be mistaken there.

As far as references to System.Collections.Immutable, there may be one such package reference in the test assembly itself. However,

<package id="System.Collections.Immutable" version="1.3.1" targetFramework="net462" />

I started with .NET Framework 4.6.2 as a nominal starting point, but I'm not attached there if need be. That to say, I could change the Target framework for the unit test assembly? To 4.7.2? Or even Core?

[rainersigwald]

Try updating to package version 1.5.0. That delivers System.Collections.Immutable, Version=1.2.3.0, which is what MSBuild (and VS 15.8) depend on.

[mwpowellhtx]

That fixed it, thank you; I updated the *Microsoft.Build.** packages, for starters. Will make sure my packages are updated flush as needed.

[rainersigwald]

Please do look at https://docs.microsoft.com/en-us/visualstudio/msbuild/updating-an-existing-application -- using Locator should help avoid these problems in the future, by ensuring you/your tests are using the same MSBuild that VS would on a given machine.

[mwpowellhtx]

@rainersigwald Let me ask you this question following up; if I updated my package references from 15.7 15.8, does this mean that anyone rebuilding and running the same tests also needs to be on the same VS version? My take away's from that lead me to wonder if a bit of packaging/DLL hell isn't the natural consequence of this approach. Not to say package delivery isn't worthwhile; but it does carry with it a massive dependency implication. Is this an accurate assessment? Or is this the problem the Locator service attempts to solve?

[rainersigwald]

That's exactly the problem the Locator is trying to solve. Prior to Visual Studio 2017, MSBuild was installed in the Global Assembly Cache, so any application that attempted to load it would get the same version. VS 2017 supports side-by-side installs, so MSBuild isn't in the GAC any more (so your Preview-channel MSBuild doesn't stomp on your release-channel MSBuild). That's mostly good, but it means that API clients have to do more work to get a consistent view of the world: either package everything that MSBuild needs (essentially impossible; it's not all open source and it's impossible to know what VS extensions someone may have installed) or locate and use the copy of MSBuild from their VS installation (which might mean picking among VS installations).

[mwpowellhtx]

@rainersigwald Gotcha, thanks very much for the clarification. :+1: