Feature request: Support DependencyContext.Default for single file applications

[0xced]

As of Microsoft.Extensions.DependencyModel 6.0.0, getting the DependencyContext of an application deployed as a single-file is not supported. The code is properly annotated with the RequiresAssemblyFiles attribute. The goal of this request is to remove the RequiresAssemblyFiles attribute and make DependencyContext.Default work for apps published in the single file deployment model instead of returning null.

I have dedicated a project to experiment with creating a DependencyContext in single-file applications: https://github.com/0xced/SingleFileAppDependencyContext

I came to the conclusion that the CoreCLR should probably expose some information about the single-file bundle structure. This would enable Microsoft.Extensions.DependencyModel to access the bundled .deps.json file in order to construct a valid DependencyContext instance.

I'd be happy to attempt a pulI request but I have never played with the CoreCLR codebase. So I'll happily accept guidance on how to best expose the single-file bundle structure through QCall, assuming that it's the best way to do it.

[ghost]

Tagging subscribers to this area: @agocke, @vitek-karas, @vsadov See info in area-owners.md if you want to be subscribed.

Issue Details

---

As of [Microsoft.Extensions.DependencyModel 6.0.0][1], getting the DependencyContext of an application deployed as a [single-file][2] is not supported. The code is [properly annotated][3] with the `RequiresAssemblyFiles` attribute. The goal of this request is to remove the `RequiresAssemblyFiles` attribute and make `DependencyContext.Default` work for apps published in the single file deployment model instead of returning `null`. I have dedicated a project to experiment with creating a `DependencyContext` in single-file applications: https://github.com/0xced/SingleFileAppDependencyContext I came to the conclusion that the CoreCLR should probably expose some information about the single-file bundle structure. This would enable Microsoft.Extensions.DependencyModel to access the bundled `.deps.json` _file_ in order to construct a valid `DependencyContext` instance. I'd be happy to attempt a pulI request but I have never played with the CoreCLR codebase. So I'll happily accept guidance on how to best expose the single-file bundle structure through QCall, assuming that it's the best way to do it. [1]: https://www.nuget.org/packages/Microsoft.Extensions.DependencyModel/6.0.0 [2]: https://docs.microsoft.com/en-us/dotnet/core/deploying/single-file/overview [3]: https://github.com/dotnet/runtime/blob/v6.0.5/src/libraries/Microsoft.Extensions.DependencyModel/src/DependencyContext.cs#L53-L54

Author:	0xced
Assignees:	-
Labels:	`area-Single-File`
Milestone:	-

[vitek-karas]

Can you please describe what is the scenario you want to solve with the DependencyContext?

We've discussed this internally a bit and the one scenario we know of is basically "Enumerate all assemblies in the app" (typically for something like dynamic service discovery in DI or similar scenarios). For that DependencyContext is not the right solution, since it's tightly coupled with the .deps.json file format. There are .NET form factors which don't have .deps.json, for example Native AOT, and pretty much all of the mobile platforms.

We would also like to treat .deps.json as an implementation detail more and more. For example, in self-contained single-file there's basically no use for that file - it just makes the output a bit larger and costs perf at startup, but it doesn't really bring any value to most apps. So ideally we should be able to completely remove it in that case.

[0xced]

We've discussed this internally a bit and the one scenario we know of is basically "Enumerate all assemblies in the app" (typically for something like dynamic service discovery in DI or similar scenarios).

Yes, that's exactly the use case: enumerate all available assemblies (even if not already loaded) and then call some methods through reflection. The Serilog.Settings.Configuration package does exactly this. It uses the DependencyContext to search for assemblies containing Serilog in the name but it doesn't work when deployed as single-file. We have discussed this in serilog/serilog-settings-configuration#304

If DependencyContext is not the right way to go then this issue becomes a duplicate of #57714 which asks for the same requirement: get a list of all known assemblies.

What would be the path to the right solution if not through DependencyContext/.deps.json ?

[vitek-karas]

This is just my thinking right now, it needs a design discussion (and API discussion):

• Introduce new public API which does this - enumerate assemblies in the app - but what does it exactly mean?:
  • What if I Assembly.LoadFrom a new file which is not part of the app, should the API now return that as well?
  • Is it guaranteed to be a stable list (as in it never changes during the lifetime of the app)
  • How does it apply at high-level to the different form factors - one such definition could be, it's the list of assemblies which make up an app at the point of dotnet publish command (or similar), which is very close to what happens today with .deps.json, but extend that to other form factors. But there could be other definitions as well.
• Based on the above API definition, go and define what it means in detail for each of the form factors we support (CoreCLR, Native AOT, single-file, mono, Android, iOS, Blazor, ...)
  • For example, should custom hosts (CoreCLR/Mono) have the ability to influence the list somehow
  • Or another - what happens for framework dependent apps - are the framework assemblies part of the list or not? (today with .deps.json they are not, but maybe they should be)
• Implement it at least in some of the form factors - probably CoreCLR, single-file, NativeAOT should be a must and then also look into the other mono based ones eventually.

The technical implementation for single-file would probably require some changes to the host/runtime interface, since the information is in the host, not in the runtime itself. It's not that hard to do, just touches several components.

Note on Native AOT: The notion of Assembly in Native AOT is a bit weird since it only makes sense for apps which use reflection. So the API may not work in some cases - which is OK, it's just an interesting twist.

[0xced]

Those are all good points. Please keep us updated in this issue (or in #57714) when the team settles on an API so that we can try it in real-world apps.

[vitek-karas]

Unfortunately I can't make any promises as to when this will happen. So far there hasn't been that many requests on this. But maybe we'll gather more over time.

[agocke]

I thought about this a little bit before and didn't really come up with a workable model for NativeAOT. Since NativeAOT needs to trim away code, keeping all the assemblies around isn't really possible or practical. Effectively, the form factor demands some sort of static specification of what's used.

@0xced Is there some sort of manual configuration alternative to Serilog that avoids scanning all the assemblies?

[0xced]

Yes, a new method that takes an explicit array of assemblies has been added to support the single-file deployment model in serilog/serilog-settings-configuration#310.

It's not yet released as a stable version but should be available in Serilog.Settings.Configuration 3.3.1.

[0xced]

Apparently there's more than enumerate all assemblies in the app use cases out there. In Carter, for example, the Library and Dependency types from Microsoft.Extensions.DependencyModel are used:

private static bool IsReferencingCarter(Library library)
{
    return library.Dependencies.Any(dependency => dependency.Name.Equals(carterAssemblyName));
}

private static bool IsReferencingFluentValidation(Library library)
{
    return library.Dependencies.Any(dependency => dependency.Name.Equals(fluentValidationAssemblyName));
}

This can't work with single-file deployment since the DependencyContext is null in the first place. Note that I'm not familiar with Carter at all, I just saw CarterCommunity/Carter#291 appear as a linked issue two days ago.

We might want to page in @jchannon who wrote this piece of code.

[vitek-karas]

There is a scenario with .deps.json which is probably never going to work in single-file. If asked for .deps.json can store compile environment, so things like defines, compiler options and... reference assemblies used to compile the assembly. This is used in scenarios where the assembly in question should be recompiled at runtime (ASP.NET can do this in some cases, where you just edit the source file and it recompiles on the fly). This will not work with single-file, for several reasons, for example there's no API to get to those reference assemblies (they can't be loaded by runtime) - and we don't have any plans to add this right now.

I don't know if this is the use case for Carter, but reference assemblies are currently not included in the single-file bundle and there's no planned effort to enable that.

[kant2002]

Partially related https://github.com/dotnet/runtime/pull/70934. I discover this issue when playing with Silk.NET + NativeAOT. Based on what I'm reading, call to CodeBase is redundant in .NET context, so hopefully that's not something overboard. Maybe that's not affects OP, but hopefully step in proper direction at least partially.

[jchannon]

That is something I considered for Carter (https://github.com/CarterCommunity/Carter/issues/291) and it could be an answer, my guess is that would slow the startup time of Carter if I'm calling Assembly.Load. I could narrow down the assemblies it loads by not loading assemblies starting with Microsoft/System. @agocke @vitek-karas does the DependencyContext.RuntimeLibraries list call Assembly.Load under the hood already so the perf differences would be minimal? cc @0xced

[jchannon]

cc @davidfowl

[jchannon]

Just to clarify I think Carter is doing the same as the above Serilog example and that it’s trying to find assemblies that reference Carter and therefore it can use those assemblies to find Carter types/interfaces inside them and register them in DI. I could get rid of the DependencyModel altogether and just get the user to pass in a string array of assemblies to use but it’s not as elegant for the user

[davidfowl]

Just FYI none of these very common application patterns are linker friendly, scanning all assemblies or relevant assemblies is problematic for trimming, this is one of the reasons the app frameworks like ASP.NET Core avoid it now. MVC isn't trim safe yet but is inching towards this. Today we scan assemblies at build time, find the relevant ones and add them to the main assembly via an attribute.

Then we use those assemblies to scan get the closure https://github.com/dotnet/aspnetcore/blob/e6dd3946f1f14e4182c70ac532dda10dd0f25f4b/src/Mvc/Mvc.Core/src/ApplicationParts/ApplicationPartManager.cs#L75-L103. This isn't trim friendly but it's more efficient.

PS: Truly dynamic plugin systems don't count, they will never be trimmable.

[vitek-karas]

I personally would also prefer build-time solution as the default for these cases. I know that some users will want truly dynamic behavior (I drop in a file and it gets picked up), but that should be opt-in in my opinion. Lot of things in .NET SDK make a soft assumption that it can see the entire app during build - especially the newer features (single-file, trimming, NativeAOT, ...), so adding that as a default assumption to libraries doesn't seem too bad.

[davidfowl]

I would too but people have been building libraries like this for years on .NET and source generators are new and not as approachable as this. It'll get easier but there's a big learning curve shifting to this model and restrictions as well.

[vitek-karas]

You're absolutely right. I meant my comment as "going forward it would be really good to use built-time solutions", which I didn't make very clear.