search

microsoft / MSBuildLocator

An API to locate MSBuild assemblies from an installed Visual Studio location. Use this to ensure that calling the MSBuild API will use the same toolset that a build from Visual Studio or msbuild.exe would.
Other
218 stars 83 forks source link

Should MSBuildLocator handle loading of NuGet assemblies? #86

Closed rynowak closed 3 years ago

rynowak commented 4 years ago

Using this with .NET sdk 3.1.X - I get errors like the following if I run any build targets.

Build FAILED.

/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: The "ProcessFrameworkReferences" task failed unexpectedly.
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.4.0.2, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified.
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: File name: 'NuGet.Frameworks, Version=5.4.0.2, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ProcessFrameworkReferences.ExecuteCore()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
    0 Warning(s)
    1 Error(s)

I think that's happening here is that the tasks assembly Microsoft.NET.Build.Tasks.dll has references to NuGet's assemblies. If this were running as part of a dedicated msbuild process it would be fine because these would be in the load context. However since I'm hosting msbuild in my process they are not.

I was able to work around this with my own assembly resolve handler, but it doesn't feel good to maintain a lot of assemblies that msbuild is usually responsible for providing.

rainersigwald commented 4 years ago

Yes, this should be built into Locator. The MSBuild-NuGet relationship is only getting closer over time; we should treat it as part of MSBuild in this context.

bettinaheim commented 4 years ago

@rainersigwald We are running into the same issue here. A fix for this would really be awesome!

rainersigwald commented 4 years ago

In microsoft/msbuild#5241, @zivkan notest that this fails only on Core, which is extra funky.

I'll sync with @microsoft/msbuild-maintainers on scheduling this in the next sprint planning (late this week? I think).

cdmihai commented 4 years ago

Doesn't this problem generalize to more than just nuget? Wouldn't any dll that MSBuild redistributes also need handling in locator? Or, even more generally, any dll that MSBuild loads and isn't part of some standard BCL path probing path, needs handling in the locator? dotnet dll loading and probing logic was always a big confusing mess to me. :)

rynowak commented 4 years ago

I think that's true, but the nuget dlls are highly likely to have breaking changes between versions 😆. Something like JSON.NET (for example) doesn't typically make binary breaking changes so it's less crucial to use the exact version that MSBuild ships.

rynowak commented 4 years ago

I've now also encountered the same issue with Microsoft.DotNet.PlatformAbstractions. NBGV loads LibGit2Sharp which loads Microsoft.DotNet.PlatformAbstractions

rainersigwald commented 4 years ago

For Core, we should think about having a universal fallback to next-to-MSBuild. That would catch a bunch of SDK-delivered stuff.

However

NBGV loads LibGit2Sharp which loads Microsoft.DotNet.PlatformAbstractions

I don't see this disassembling LibGit2Sharp 0.25.0.0 that comes with NBGV 2.2.13. How did you figure that out?

rynowak commented 4 years ago

The error message told me! I think I sent the exact error message to you in teams a few days ago. Are you saying that your chat window is not an issue tracker? 😆 

[4/17 3:26 PM] Ryan Nowak

/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: The "Nerdbank.GitVersioning.Tasks.GetBuildVersion" task failed unexpectedly.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:  ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: File name: 'Microsoft.DotNet.PlatformAbstractions, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitLoaderContext.GetNativeLibraryDirectory()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitLoaderContext.LoadUnmanagedDll(String unmanagedDllName)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at System.Runtime.Loader.AssemblyLoadContext.ResolveUnmanagedDll(String unmanagedDllName, IntPtr gchManagedAssemblyLoadContext)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.InitializeNativeLibrary()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods..cctor()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    --- End of inner exception stack trace ---
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_libgit2_opts(Int32 option, UInt32 level, String path)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.Proxy.git_libgit2_opts_set_search_path(ConfigurationLevel level, String path)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.GlobalSettings.SetConfigSearchPaths(ConfigurationLevel level, String[] paths)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitExtensions.OpenGitRepo(String pathUnderGitRepo, Boolean useDefaultConfigSearchPaths)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.VersionOracle.Create(String projectDirectory, String gitRepoDirectory, ICloudBuild cloudBuild, Nullable`1 overrideBuildNumberOffset, String projectPathRelativeToGitRepoRoot)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.Tasks.GetBuildVersion.ExecuteInner()
ErikSchierboom commented 4 years ago

Just wanted to let you know that I have the same issue :)

Yes, this should be built into Locator. The MSBuild-NuGet relationship is only getting closer over time; we should treat it as part of MSBuild in this context.

Looking forward to having this built-in!

xsacha commented 4 years ago

As a workaround, I'm using global.json with:

{
  "sdk": {
    "version": "3.1.100"
  }
}
zivkan commented 4 years ago

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};
AArnott commented 4 years ago

@rynowak @rainersigwald The Microsoft.DotNet.PlatformAbstractions.dll is supposed to ship with nerdbank.gitversioning.dll itself, but didn't because msbuild used to include it I guess and doesn't any more. Anyway, this was tracked by https://github.com/dotnet/Nerdbank.GitVersioning/issues/466 and is fixed now.

Gambero81 commented 4 years ago

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};

@zivkan i tried this workaround without success..

C:\Program Files\dotnet\sdk\3.1.401\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets: (59, 5): errore imprevisto dell'attività "ProcessFrameworkReferences". System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.7.0.7

In the "C:\Program Files\dotnet\sdk\3.1.401" folder i have NuGet.Frameworks.dll version 5.7.0.6 but it search for 5.7.0.7!

ricardo-espinoza commented 4 years ago

We encountered the same issue in a component of the Microsoft Quantum Development Kit. I also followed @zivkan 's workaround as shown in the pull request below: https://github.com/microsoft/qsharp-compiler/pull/566

CAlexGonzalez commented 3 years ago

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};

I got the same issue: C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: The "ProcessFrameworkReferences" task failed unexpectedly. C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.8.0.6930, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified. C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: File name: 'NuGet.Frameworks, Version=5.8.0.6930, Culture=neutral, PublicKeyToken=31bf3856ad364e35' C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.NET.Build.Tasks.ProcessFrameworkReferences.ExecuteCore() C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) 0 Warning(s) 1 Error(s)

Thanks Andy, your workaround also solved my issue.

BjarkeCK commented 3 years ago

A heads up to anyone maybe struggling to get the workaround working.

static void Main(string[] args)
{
    var instance = MSBuildLocator.RegisterDefaults();
    AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
    {
        var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
        if (File.Exists(path))
        {
            return assemblyLoadContext.LoadFromAssemblyPath(path);
        }

        return null;
    };

    Console.WriteLine(BuildManager.DefaultBuildManager); // This will prevent it from compiling.
}

That fails to compile, and gives me the following error:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified. File name: 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' at WatchRun.Program.Main(String[] args)

But if you simply make sure the only thing you do in main is to register the resolver, and put everything else in a nother method. Then it will compile and the workaround works.

static void Main(string[] args)
{
    var instance = MSBuildLocator.RegisterDefaults();
    AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
    {
        var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
        if (File.Exists(path))
        {
            return assemblyLoadContext.LoadFromAssemblyPath(path);
        }

        return null;
    };

    NowItWorks();
}

private static void NowItWorks()
{
    Console.WriteLine(BuildManager.DefaultBuildManager);
}