Open vitek-karas opened 5 years ago
I've also run into this problem, and believe we need to add a public LoadFromUnmanagedDllName method to AssemblyLoadContext.
But there's a deeper issue.
ALCs are essential for isolation & dynamic loading because we no longer have AppDomains.
However, isolation & dynamic loading is (almost) useless if it relies on libraries being written in a special way to make them "isolation-friendly" or "dynamic-load-friendly": Even if you write an assembly specifically to be isolated, chances are that you'll have at least one NuGet dependency that hasn't been written specially to be isolated.
At a minimum, one should fall into the pit of success when using .NET Core in a canonical fashion. A library that you write should execute correctly in a non-default ALC without special programming.
The difficulty with NativeLibrary.Load (when called with a simple name) is that it bypasses the resolution logic of the executing assembly's ALC, breaking the ALC isolation model. It's likely that any library that uses NativeLibrary.Load (e.g, Microsoft.Data.Sqlite, which calls SQLitePCLRaw.nativelibrary), will break when loaded into a non-default ALC or when loaded dynamically. Ironically, a major purpose of NativeLibrary is to allow the name of the native library to be chosen at runtime (i.e., dynamically). And yet, when its assembly that calls NativeLibrary is loaded dynamically (into a custom ALC), it breaks!
In fixing this, I believe there are two important principles:
Here's what I propose:
I appreciate that there are subtle issues with Assembly.Load(string) inferring the ALC (which have led to ContextualReflectionContext). But the problem stems from the ALC being implicit rather than explicit; not from the wrong default choice of ALC. If we were to change Assembly.Load so that it always used the default ALC, the situation would get far worse. So given that we're stuck with Assembly.Load(string):
NativeLibrary.Load (string) is analogous. The ALC is also implicit rather than explicit. Like Assembly.Load(string), we can't take it away. The issue is, what is the best default choice for the inferred ALC? The situation is
I understand that NativeLibrary was intended as a low-level API. But what does this mean? It's just another way of saying that it doesn't invoke the current ALC for resolution (begging the question) and also that it's technically valid to call NativeLibrary from an ALC's LoadUnmanagedDll method. But why would you do this, when you already have LoadUnmanagedDllFromPath right there in front of you - along with examples that tell you to call LoadUnmanagedDllFromPath?
Note that this entire issue applies to when you're calling NativeLibrary.Load with a simple name (in other words, when you're asking it to resolve). In this situation, the library author is taking responsibility for what DLL to load, while allowing the application to determine where the DLL is located.
This should be fixed in .NET Core 5, before too many API authors start using NativeLibrary. Each time they do, they're unintentionally writing a library that will break in a non-default ALC.
NativeLibrary.Load(string)
is a low-level API that wraps the underlying library load OS API and nothing else. It is meant to be used e.g. for loading OS and similar libraries that are outside the application.NativeLibrary.Load(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
is a higher level API. The intention for this API was to behave as if DllImport was used in the given assembly.The problem seems to be that NativeLibrary.Load(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
does not behave as DllImport in the given assembly. I believe that the cleanest fix may be to change NativeLibrary.Load(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
to call AssemblyLoadContext.LoadUnmanagedDll
and AssemblyLoadContext.ResolvingUnmanagedDll
. No new API needed. Thoughts?
It is not possible to do the same for native library resolution. While the child can overload ChildALC.LoadUnmanagedDll, there's no API it can call on the ParentALC to fall back to its native library resolution.
The native library is an implementation detail of an assembly. Thus the ALC that actually loaded the assembly should be the one responsible for loading its native libraries too. What is a real example where the two ALC are different and the chaining is needed?
The chaining problem and the problem that @albahari desribed do not seem to be same issues.
I agree that the chaining problem is not the same as @albahari states - they're just related as they are around the same APIs. And if we're considering adding/modifying public APIs, it's worth to have most of the related issues around it together.
The chaining problem is a hypothetical scenario where there are two levels of plugins. AssemblyLoadContext
does not naturally create hierarchy - all ALCs are peers except for Default
which is "parent" for everything else. Because of this, for the multi-level plugin scenario the custom ALCs need to manually create the hierarchy, by calling the "parent" from the "child" explicitly. This is possible for managed assemblies as the necessary APIs are public. It's currently not possible for native libraries.
That said I do agree that this should be a niche case - as you mention in vast majority of cases the native dependencies of an assembly should be resolved by the ALC into which the assembly is loaded (as that is logically how it is constructed). So this mostly comes into play for explicit loads like the NativeLibrary.Load(string, Assembly, DllImportSearchPath?)
(once it's fixed) - or for "weird" scenarios where the plugins want to explicitly share native dependencies as well.
I believe that the cleanest fix may be to change NativeLibrary.Load(string libraryName, Assembly assembly, DllImportSearchPath? searchPath) to call AssemblyLoadContext.LoadUnmanagedDll and AssemblyLoadContext.ResolvingUnmanagedDll. No new API needed. Thoughts?
Yes, that would be a big step forward and solve at least half the problem.
But an issue would remain: it's far too easy to make a mistake and call NativeLibrary.Load("foo") instead of NativeLibrary.Load("foo", Assembly.GetExecutingAssembly(), null) or NativeLibrary.Load("foo", typeof(MyWrapper).Assembly, null). The difficulty is that unless the library author tests their code outside the default ALC, they'll never know that there's a problem.
I can see two ways to fix the remaining issue:
(2) is cleaner, but more likely to break existing code.
NativeLibrary.Load(string)
behave as intended, and only resolve OS and libraries that are outside the application.
The intended behavior of NativeLibrary.Load
is to just call the underlying OS API. On Unix, the native library probing paths do not include the program directory by default. On Windows, the native library probing paths do include the .exe directory by default. This API is not trying to hide OS differences.
In that case, I misunderstood: it's not looking at the default ALC's TPA paths, just the .exe directory.
So then your suggestion of changing NativeLibrary.Load(string libraryName, Assembly assembly, DllImportSearchPath? searchPath) to call the assembly's AssemblyLoadContext.LoadUnmanagedDll and AssemblyLoadContext.ResolvingUnmanagedDll would work nicely.
Should I log a new issue for this?
i.e., changing the behaviour of NativeLibrary.Load(string,Assembly,DllImportSearchPath)?
@albahari Feel free to do that - I think in the end we might do only that - since that would more or less fix the issue mentioned in this one...
anyone hits the Managed Assemblies's Alc across issue ; in my library https://github.com/John0King/LazyMan.ModularLoader, I want to side load the whole web site as subapp , then I facing this Assemblies's Alc across issue, it too complex, that I can not reproduce this issue simple, but when you use my lib to run, there are at least 98% web app will crash.
a little explain: (this is after a long time debug)
Microsoft.AspNetCore.App
assemblies is load in default ALC by default , in this case my lib worksMicrosoft.AspNetCore.App
assemblies will in web app's local folder, so those .dll
well load in pluginAlc
, notice: not all will copy to local folder.Microsoft.AspNetCore.App
's assemblise, so the issue happend. Microsoft.Extensions.DependencyInject.Abstraction.dll
in pluginAlc , let's called p_core.dll for shot , and core.dll
for those in default alc) because p_core.dll is in pluginAlc, and some DI configure call in default alc, , now core.dll
call will looking p_core.dll
's class in default alc, that cause MethodNotFoundException
I can't , because the AssemblyDependencyResolver
can not located the core.dll
core.dll
in default Alc?I can't by default , because it in the .deps.json
, and will be discovered by AssemblyDependencyResolver
run https://github.com/John0King/LazyMan.ModularLoader/tree/master/test/SubAppTest the HostApp, and copy a bigger webapp (just not black app) for example OrchardCore (need to change the CreateWebHost
to CreateWebHostBuilder
)
@John0King Is your problem related to this issue? As far as I can tell you don't have any native components in your project. This issue is only about problems with handling native dependencies in hierarchical ALCs - neither of which you seem to be using. It would be better to create a new issue for this.
That aside I think that trying to load ASP.NET app into a secondary ALC is going to be problematic. I've seen issues around this already. Mostly they're because ASP.NET (and the libraries it uses) are not ALC aware and sometimes do things which will lead to problems. Good example is here: https://github.com/dotnet/runtime/blob/e25517ea27311297c1e3946acb3b4382d5fa7fef/src/libraries/Microsoft.Extensions.Hosting/src/Host.cs#L82 - since Microsoft.Extensions.Hosting
is going to be loaded to the default ALC (it's part of a framework) the Assembly.Load
will work on the default ALC only - so if you try to load an application into a secondary ALC this will either directly fail, or it will not work as expected. I'm not sure how many other samples are there in ASP.NET of this.
There might be some tricks you can play with CurrentContextualReflectionContext
but it's not guaranteed to fix the problem. In short I think this is a problem in ASP.NET which was simply not designed to work in secondary ALCs - at least yet.
I'm sorry ask here, but I really do not know where should I create a new issue, runtime/aspnetcore , the reason I post here is because I think it should be a alc fullback issue:
defualt alc: A --(call) --> B --(call)---> C
plugin alc : D--(call)-> [ A ](fallback from default alc) --🚫--> C ′
in plugin alc, assembly D in plugin alc can call assembly A from defualt alc , but then it can not call the C ′ from plugin alc, because the A is in default alc, and it can only find C instead of C′,
the theory to solve this, is make the type/assembly finding by CurrentContextualReflectionContext
so when D
call A
it is in plugin alc Context, and when find C , it will look plugin alc first
Currently it's possible to create a "chain" of
AssemblyLoadContext
instances such that there's aParentALC
andChildALC
.ChildALC
has a reference toParentALC
and every time theChildALC.Load
is called, if it can't resolve the assembly it falls back to parent by callingParentALC.LoadFromAssemblyName
(which will in turn callParentALC.Load
and so on).It is not possible to do the same for native library resolution. While the child can overload
ChildALC.LoadUnmanagedDll
, there's no API it can call on theParentALC
to fall back to its native library resolution. There might be some workarounds using contextual reflection andNativeLibrary
but that would be rather tricky and absolutely non-discoverable.Given the current API design, it would probably be cleaner to add a new API to
NativeLibrary
- something likeIntPtr NativeLibrary.Load(string libraryName, AssemblyLoadContext context)
. Or alternatively expose something similar on the ALC itself.