Loading Newtonsoft.Json in Cake.CoreCLR throws during assembly loading

What You Are Seeing?

Loading Newtonsoft.Json in Cake.CoreCLR throws during assembly loading.

What is Expected?

Rainbows and sunshine.

What version of Cake are you using?

Cake.CoreCLR 0.26.1

Are you running on a 32 or 64 bit system?

What environment are you running on? Windows? Linux? Mac?

Windows

How Did You Get This To Happen? (Steps to Reproduce)

Run the following build.cake:

#addin nuget:?package=Newtonsoft.Json&version=11.0.2

Task("Default")
    .Does(() =>
    {
    });

RunTarget("Default");

Raw assembly loading also fails:

#r "E:\Code\discoverdotnet\tools\Addins\Newtonsoft.Json.11.0.2\lib\netstandard2.0\Newtonsoft.Json.dll"

Task("Default")
    .Does(() =>
    {
    });

RunTarget("Default");

Output Log

E:\Code\discoverdotnet>build -target preview -verbosity diagnostic
  Writing C:\Users\dglick\AppData\Local\Temp\tmp390D.tmp
info : Adding PackageReference for package 'cake.coreclr' into project 'E:\Code\discoverdotnet\tools\build.csproj'.
log  : Restoring packages for E:\Code\discoverdotnet\tools\build.csproj...
info : Package 'cake.coreclr' is compatible with all the specified frameworks in project 'E:\Code\discoverdotnet\tools\build.csproj'.
info : PackageReference for package 'cake.coreclr' version '0.26.1' updated in file 'E:\Code\discoverdotnet\tools\build.csproj'.
Module directory does not exist.
NuGet.config not found.
Analyzing build script...
Analyzing E:/Code/discoverdotnet/build.cake...
Processing build script...
Installing addins...
Found package 'Newtonsoft.Json 11.0.2' in 'E:/Code/discoverdotnet/tools/Addins'.
Package Newtonsoft.Json.11.0.2 has already been installed.
Successfully installed 'Newtonsoft.Json 11.0.2' to E:/Code/discoverdotnet/tools/Addins
Executing nuget actions took 35.23 ms
The addin Newtonsoft.Json will reference Newtonsoft.Json.dll.
Error: System.IO.FileLoadException: Could not load file or assembly 'Newtonsoft.Json, Version=11.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'.
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile)
   at Cake.Core.Polyfill.AssemblyHelper.LoadAssembly(ICakeEnvironment environment, IFileSystem fileSystem, FilePath path) in E:\Code\cake\src\Cake.Core\Polyfill\AssemblyHelper.cs:line 49
   at Cake.Core.Reflection.AssemblyLoader.Load(FilePath path, Boolean verify) in E:\Code\cake\src\Cake.Core\Reflection\AssemblyLoader.cs:line 31
   at Cake.Core.Scripting.ScriptRunner.Run(IScriptHost host, FilePath scriptPath, IDictionary`2 arguments) in E:\Code\cake\src\Cake.Core\Scripting\ScriptRunner.cs:line 171
   at Cake.Commands.BuildCommand.Execute(CakeOptions options) in E:\Code\cake\src\Cake\Commands\BuildCommand.cs:line 34
   at Cake.CakeApplication.Run(CakeOptions options) in E:\Code\cake\src\Cake\CakeApplication.cs:line 46
   at Cake.Program.Main() in E:\Code\cake\src\Cake\Program.cs:line 82

It's not totally clear why this is happening. Suspicion is that it has something to do with the .NET SDK already containing a custom build of Newtonsoft.Json and the assembly load conflicting with that.

When running with COREHOST_TRACE I see this pretty early on:

Processing TPA for deps entry [Newtonsoft.Json, 9.0.1, lib/netstandard1.0/Newtonsoft.Json.dll]
  Considering entry [Newtonsoft.Json/9.0.1/lib/netstandard1.0/Newtonsoft.Json.dll] and probe dir [C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.0.6]
    Skipping... probe in deps json failed
    Skipping... not found in probe dir 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.0.6'
  Considering entry [Newtonsoft.Json/9.0.1/lib/netstandard1.0/Newtonsoft.Json.dll] and probe dir []
    Local path query exists C:\Program Files\dotnet\sdk\2.1.103\Newtonsoft.Json.dll
    Probed deps dir and matched 'C:\Program Files\dotnet\sdk\2.1.103\Newtonsoft.Json.dll'
Adding tpa entry: C:\Program Files\dotnet\sdk\2.1.103\Newtonsoft.Json.dll

(TPA = trusted platform assemblies)

So it looks like the version of JSON.NET that ships with .NET Core SDK 2.1.103 is 9.0.1 (located at "C:\Program Files\dotnet\sdk\2.1.103\Newtonsoft.Json.dll"). A little further down I also see this:

Adding runtime asset lib/netstandard1.0/Newtonsoft.Json.dll from Newtonsoft.Json/9.0.1

And then finally this:

Reconciling library Newtonsoft.Json/9.0.1
Parsed runtime deps entry 28 for asset name: Newtonsoft.Json from package: Newtonsoft.Json, version: 9.0.1, relpath: lib/netstandard1.0/Newtonsoft.Json.dll

So the real question is: why does Cake have a problem loading a different version at runtime? Clearly other .NET Core apps can bind to a different version, otherwise there'd be a major outcry given how popular the library is.

Starting to think that Newtonsoft.Json being in the TPA list is a red herring and doesn’t really have anything to do with this.

Instead, my new theory is that this is a case of the Cake AssemblyLoader being late to the party. Because Cake.Core references Microsoft.Extensions.DependencyModel which in turn references Newtonsoft.Json, I think Cake.Core is probably binding to Newtonsoft.Json before the runtime NuGet package installation and assembly losing has a chance to handle user preprocessor directives. That means whatever version of Newtonsoft.Json shipped with Cake.Core is going to get into the AppDomain first and the runtime assembly load will fail if it tried to load the same assembly with a different version.

This is t a problem if an addin or tool uses a version of that assembly lower than the one Cake is binding to directly, but if the addin or tool needs a higher version it’s going to have a bad time. One possible solution could be to hook assembly resolution and return whatever assemblies were already loaded regardless of the version - basically doing binding redirects to the previously loaded version. That could work as long as the addin or tool doesn’t require functionality in the higher version.

I’ll continue to look at this tomorrow, but wanted to get some thoughts down in case anyone had other ideas or feedback.

@daveaglick yes it’s a transitive dependency. It’s at least loaded via Cake.NuGet which depends on NuGet clients libs, which depends on NewtonSoft.Json.

Wonder if adding a binding redirect to app.config fixes this issue?

E.g. For Newtonsoft.Json (support a large range of versions). <bindingRedirect oldVersion="0.0.0.0-99.99.99.99" newVersion="9.0.1"/>

Here’s the transitive dependency. https://www.nuget.org/packages/NuGet.Protocol/4.6.0

@mholo65 But Cake.CoreCLR is executed via dotnet cake.dll ... so app.config isn't in play. I could hand edit the Cake.deps.json, but that seems really hacky and hard to maintain.

I got curious about which assemblies are loaded into the script host context at runtime:

using System.Reflection;

Task("Default")
    .Does(() =>
    {
            foreach(Assembly a in AppDomain.CurrentDomain.GetAssemblies())
            {
                if(!a.IsDynamic)
                {
                    Information(a.FullName);
                }
            }
    });

RunTarget("Default");

Here's the full list:

System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
Cake, Version=0.26.1.0, Culture=neutral, PublicKeyToken=null
System.Runtime, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Cake.Core, Version=0.26.1.0, Culture=neutral, PublicKeyToken=null
Autofac, Version=4.6.2.0, Culture=neutral, PublicKeyToken=17863af14b0044da
netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
System.Linq, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Cake.Common, Version=0.26.1.0, Culture=neutral, PublicKeyToken=null
Cake.NuGet, Version=0.26.1.0, Culture=neutral, PublicKeyToken=null
System.Runtime.Extensions, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Collections, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Console, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Threading, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Collections.Concurrent, Version=4.0.14.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.ComponentModel, Version=4.0.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Globalization, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Linq.Expressions, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection.Emit.ILGeneration, Version=4.0.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection.Primitives, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection.Emit.Lightweight, Version=4.0.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection.Extensions, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Runtime.InteropServices.RuntimeInformation, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.IO.FileSystem, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Runtime.InteropServices, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.ObjectModel, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
NuGet.Frameworks, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
NuGet.Common, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.Threading.Tasks, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
NuGet.Configuration, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
NuGet.PackageManagement, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
NuGet.Protocol, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
NuGet.Packaging, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
NuGet.Versioning, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.Resources.ResourceManager, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.IO, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Xml.ReaderWriter, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Private.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
System.Xml.XDocument, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Private.Xml.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
System.Threading.Thread, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Security.Cryptography.Algorithms, Version=4.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Security.Cryptography.Primitives, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Text.Encoding, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.IO.FileSystem.Primitives, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Private.Uri, Version=4.0.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Text.Encoding.Extensions, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
NuGet.Packaging.Core, Version=4.3.0.5, Culture=neutral, PublicKeyToken=31bf3856ad364e35
Microsoft.Win32.Registry, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Text.RegularExpressions, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Net.Primitives, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Security.AccessControl, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Security.Principal.Windows, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.CodeAnalysis.Scripting, Version=2.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.Collections.Immutable, Version=1.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.CodeAnalysis, Version=2.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
Microsoft.CodeAnalysis.CSharp.Scripting, Version=2.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.Runtime.Loader, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection.Metadata, Version=1.4.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.CodeAnalysis.CSharp, Version=2.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.ValueTuple, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
System.AppContext, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Threading.Tasks.Parallel, Version=4.0.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Diagnostics.Tracing, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.IO.MemoryMappedFiles, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
R*9ef985e8-be02-44bd-b7e9-f21849e5d143#1-0, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null

But no Newtonsoft.Json! However, if I add a simple name reference to the build script:

#r Newtonsoft.Json

Boom:

Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed

Which is the version that sits alongside Cake.dll in the tools directory.

I'm pretty sure this is what's going on:

Cake ships alongside a particular version of Newtonsoft.Json.dll (9.0.0.0 for the current version of Cake)
When we try to load a NuGet Package or raw assembly for Newtonsoft.Json, the default assembly resolver first looks alongside the executable (Cake.dll) and when it finds the assembly but the version doesn't match, the assembly load fails
Because the load in the Cake setup process failed, the assembly never gets added to the references in the script in ScriptRunner so any tool or addin that needs Newtonsoft.Json will fail at script runtime

This means that:

Any tool or addin that depends on a version of Newtonsoft.Json higher than the one shipped alongside Cake will fail

Not a great situation given how popular Newtonsoft.Json is.

To try and figure out a way around it, I started playing with runtime binding redirection by hooking assembly resolution and adding this to my build script:

Setup(context =>
{
    AppDomain.CurrentDomain.AssemblyResolve += (_, eventArgs) =>
    {
        AssemblyName name = new AssemblyName(eventArgs.Name);
        Verbose($"Resolving assembly {eventArgs.Name}");
        Assembly assembly = AppDomain.CurrentDomain.GetAssemblies()
            .FirstOrDefault(x => !x.IsDynamic && x.GetName().Name == name.Name)
            ?? Assembly.Load(name.Name);       
        if(assembly != null)
        {
            Verbose($"Resolved by assembly {assembly.FullName}");
        }
        else
        {
            Verbose($"Assembly not resolved");
        }
        return assembly;
    };
});

Which solves my own problem:

Resolving assembly Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed
Resolved by assembly Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed

But still isn't ideal given that:

You still get a warning when loading a NuGet package for a tool or addin that depends on Newtonsoft.Json: Could not load E:\Code\discoverdotnet\tools\Addins\NetlifySharp.0.1.0\lib\netstandard1.6\NetlifySharp.dll (missing Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed)
If the tool or addin uses a more recent version because of breaking changes from the version shipped with Cake, there's the potential for all kinds of fun runtime problems since we essentially redirected the binding to a lower version

Not sure how to solve across the board... Maybe ship the latest and greatest version of Newtonsoft.Json with each release of Cake and add a resolver like the one above to the script boilerplate (or do the equivalent directly in Cake when evaluating the script since they share the same AppDomain)?

PR incoming. I updated the version of Newtonsoft.Json to the latest and added a runtime assembly resolver for assemblies that can't be found. This tackles the problem in two ways:

If an addin references an earlier version of Newtonsoft.Json, everything just works because the default resolver will happily bind to a later version - and since Cake now ships with the latest version, this should cover most cases (will need to remember to keep it up to date)
If a new version of Newtonsoft.Json is published and referenced by an addin that's later than what Cake ships with, the new runtime assembly resolver will step in and bind to the lower version

Looks like the only version of Newtonsoft.Json package available for scripts is 11.0.2 as of now. Is there a way to pick another version of this package as a build script addin?

cake-build / cake