Author: | linkem |
---|---|
Assignees: | - |
Labels: | `area-AssemblyLoader-coreclr`, `untriaged` |
Milestone: | - |
Open linkem opened 1 year ago
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.
Tagging subscribers to this area: @vitek-karas, @agocke, @vsadov See info in area-owners.md if you want to be subscribed.
Author: | linkem |
---|---|
Assignees: | - |
Labels: | `area-AssemblyLoader-coreclr`, `untriaged` |
Milestone: | - |
The problem is that your plugin brings its own version of System.Text.Json while using parts of the shared framework which also reference it. The framework which is loaded into the Default context uses S.T.Json from the Default context. The plugin code references the S.T.Json in the plugin load context. But then it calls code from the framework (HttpClient
and so on). The problem is not very visible, it's default parameters. ReadFromJsonAsync
has several default parameters one of which is JsonSerializerOptions
which is a type from S.T.Json. The way default parameters work in .NET is that the compiler hardcodes the default at the callsite - so even though the source code looks like:
ReadFromJsonAsync<TodoModel>()
In reality the runtime sees the call as
ReadFromJsonAsync<TodoModel>((JsonSerializerOptions)null, default(CancellationToken));
The problem is that the JsonSerializerOptions
type comes from the plugin - so the version of S.T.Json from the plugin load context. And thus it doesn't match the method declared in the framework which takes that type from the S.T.Json in the defautl load context.
In short - any assembly which is part of the shared framework should not be also carried by the plugins - it will break anytime where the plugin uses APIs which refer to types from such assembly and interoperates with the default framework APIs which do the same.
So in this case, remove the PackageReference to System.Text.Json.
The reason why it works if the package version is 6.0.0: If the version is 6.0.0 the SDK realizes that the version is available in the framework and won't actually put the file into the plugin. If the version is 6.0.7 that doesn't happen and the plugin carries its own copy.
There should be no need to have a package reference to System.Text.Json - it's part of the framework, why would you bring your own copy of it.
I know this is confusing to debug, because the assemblies/types look identical while the runtime doesn't see them as the same.
@vitek-karas Thank you for quick response and very detailed summary of issue.
I know that referencing S.T.Json in net6 applications is not required and can be easily avoided. My example is intentionally over simplified to reproduce problem. In real life, our net6 applications reference netstandard libraries that reference S.T.Json. In that case net6 application will have S.T.Json .dll in output without referencing it directly(transitive reference). It's basically impossible to avoid transitive references, especially in case of third party libs and nuget packages.
I have created repro with netstandard lib here. PluginApp
don't reference S.T.Json directly but its present in output folder.
Just for record, this issue is not exclusive for S.T.Json, same thing happend with other libs.
Few more questions:
Is it intended behavior that AssemblyLoadContext
loads assemblies with duplicated names from framework and local folder at the same time? I thought that the assembly present in the local folder will only be used if the Default context doesn't have this assembly or has it in older version, but will never load assemblies with same names.
How could we avoid issues like this? What would be best solution if we can't avoid referencing netstandard packages that have transitive references to packages from framework?
The problem is versioning - the version 6.0.7 of System.Text.Json package contains the assembly with higher version than the one in the framework. This leads the SDK to keep it, instead of relying on the framework.
Note that this has nothing to do with netstandard
targetting - if you change your project to reference version 6.0.0
the System.Text.Json will not be part of the plugin output even with the netstandard library used.
I don't know what are the version resolution rules in the SDK, they seem a bit harsh for this scenario.
@dsplaisted : If an app targeting net6.0
has a package reference to System.Text.Json version 6.0.7 (latest) it will get its own private copy of the assembly, even though that version is the same as the version in the latest runtime 6.0.11. Is there some way to tell the SDK to always rely on framework provided assembly even if the versions are a bit off? (I assume the behavior is because the ref package for 6.0.0 has a lower version of the assembly).
Is it intended behavior that AssemblyLoadContext loads assemblies with duplicated names from framework and local folder at the same time? I thought that the assembly present in the local folder will only be used if the Default context doesn't have this assembly or has it in older version, but will never load assemblies with same names.
Each instance of AssemblyLoadContext
(ALC) will only load one version of a given assembly. But the important thing to realize is that ALCs are not isolation boundaries. They're only used when resolving assembly references (or explicit loads). Once assemblies are loaded ALCs are basically ignored. So you can have code which in theory has instances of the same type from two different versions of the assembly (from different ALCs) in one array (for example).
The assembly resolution for the Default ALC is governed by the app.deps.json
which is produced by the SDK. Typically this means that the Default ALC will resolve only assemblies in the main app (and only those which were part of the build of the app).
The assembly resolution for the secondary ALCs is driven by the implementation of the AssemblyLoadContext.Load
method. In your case (and typically for plugins), this uses AssemblyDependencyResolver
. That class also relies on plugin.deps.json
which is produced by the SDK. And it behaves similarly - in that it will only resolve assemblies which are part of the build of the plugin.
The "Sharing" of assemblies (in your case the plugin interface assembly) is done by not including that assembly in the output of the plugin, in which case AssemblyDependencyResolver
will not resolve it, the AssemblyLoadContext.Load
will return null
and it will fall back to the Default ALC, which will resolve it from the main app.
How could we avoid issues like this? What would be best solution if we can't avoid referencing netstandard packages that have transitive references to packages from framework?
You could hardcode that you don't allow your plugins to carry their own version of assemblies provided by the host. In the AssemblyLoadContext.Load
override call AssemblyLoadContext.Default.LoadFromAssemblyName(assemblyName)
first and if it doesn't fail, use the retuned assembly over anything returned by the AssemblyDependencyResolver
. The downside of this approach is that it requires try/catch as LoadFromAssemblyName
will throw if it can't resolve the assembly. But if you're not loading that many plugins it should not be a big issue.
/cc @elinor-fung
You could hardcode that you don't allow your plugins to carry their own version of assemblies provided by the host. In the AssemblyLoadContext.Load override call AssemblyLoadContext.Default.LoadFromAssemblyName(assemblyName) first and if it doesn't fail, use the retuned assembly over anything returned by the AssemblyDependencyResolver. The downside of this approach is that it requires try/catch as LoadFromAssemblyName will throw if it can't resolve the assembly. But if you're not loading that many plugins it should not be a big issue.
This approach is something that I consider as last resort, only if we won't find any better solution.
Implementing it this way, we will lose possibility of loading Assemblies with different version than host already have. For example, if host depends on MediatR
package in version 10.0.0
and my plugin app depends on version 11.0.0
, then all plugins will be forced to use 10.0.0
, this may break everything.
In real life scenario we have host application that have a lot of references and whole idea of using ALC was to enable our plugins to use their own versions of assemblies and override ones that host already have. Our approximate number of plugins will be around 10-15.
Is there any safe way to support scenario where host application has some assembly and plugin brings other version of same assembly?
Is there any safe way to support scenario where host application has some assembly and plugin brings other version of same assembly?
It should work just fine, the problem only occurs if the code tries to use types from such assembly to communicate across the host boundary. That typically happens for assemblies which are part of the framework (as above) or those used in the plugin "interface" - all those should not ship with the plugin, but instead should be provided by the host.
It should work just fine, the problem only occurs if the code tries to use types from such assembly to communicate across the host boundary. That typically happens for assemblies which are part of the framework (as above) or those used in the plugin "interface" - all those should not ship with the plugin, but instead should be provided by the host.
For plugin "interface" situation is quite easy, we can use construct in csproj file like this
<ProjectReference Include="..\Plugin.Abstraction\Plugin.Abstraction.csproj">
<Private>false</Private>
<ExcludeAssets>runtime</ExcludeAssets>
</ProjectReference>
but what about all framework assemblies?
We have to create hardcoded list of all assemblies that are in dotnet/shared/6.0.*/Microsoft.NETCore.App
and dotnet/shared/6.0.*/Microsoft.AspNetCore.App
, then during build/publish plugin app should remove them from output folder? (it would be nice if there was some csproj parameter that could do it).
Another way would be to use hardcoded list of assemblies in Loader and use Default context to resolve them.
Anyway, both solution looks a bit weird, maybe there are some other way to achieve it?
I agree it's not ideal - I honestly would like to see some SDK solution, because typically one would expect the SDK to resolve the framework dependencies to the framework.
There's no good way to tell if assembly belongs to the framework or not (you could use path, but that only works for framework dependent apps, and is still a bit weird) - which is kind of by design, because for self-contained app there's no "framework" really, everything is the "app".
Thank you for many tips. For now we do not consider to use self-contained apps, at least not in terms of plugins. I have created small PoC of the solution that we came up here. It looks like in this simple example its working fine, I need some more time to check it on bigger "real life" project. Could you check if my PoC is something that you had in mind (PluginLoadContext and hardcoded list of assemblies)?
That should work, it's obviously fragile - I don't think new assemblies will be added a patch release, but I wouldn't be surprised if the list looked different for .NET 7 and so on. That said it's probably the best/simplest one can do right now.
The problem is versioning - the version 6.0.7 of System.Text.Json package contains the assembly with higher version than the one in the framework. This leads the SDK to keep it, instead of relying on the framework. Note that this has nothing to do with
netstandard
targetting - if you change your project to reference version6.0.0
the System.Text.Json will not be part of the plugin output even with the netstandard library used.I don't know what are the version resolution rules in the SDK, they seem a bit harsh for this scenario.
@dsplaisted : If an app targeting
net6.0
has a package reference to System.Text.Json version 6.0.7 (latest) it will get its own private copy of the assembly, even though that version is the same as the version in the latest runtime 6.0.11. Is there some way to tell the SDK to always rely on framework provided assembly even if the versions are a bit off? (I assume the behavior is because the ref package for 6.0.0 has a lower version of the assembly).
No, there's not currently a way to do this. I've filed https://github.com/dotnet/sdk/issues/29280 to track adding a way to do so.
Description
Application loaded by
AssemblyLoadContext
(implementation from documentation) loads multiple.dll
files with sameAssemblyName
which leads to exceptions likeSystem.MissingMethodException Method not found: ...
Reproduction Steps
Small repo with reproduction code here
Repro with exception
PluginApp1
applicationPluginHost
Repro with listed duplicated assemblies
PluginApp1.PluginApp1.cs
line30
PluginApp1
applicationPluginHost
Application should print output similar to this one:
Expected behavior
Plugin application should be started without duplicated assemblies, it should use assembly that is in local folder or in
shared\Microsoft.NETCore.App
but not both.Actual behavior
In some cases duplicated assemblies are loaded.
Regression?
No response
Known Workarounds
For case presented in repro, in application
PluginApp1
package reference ofSystem.Text.Json
can be changed from6.0.7
to6.0.0
, but this is not applicable in more complex scenariosConfiguration
But it also happens on dotnet 6 runtime
Other information
No response