dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.93k stars 4.64k forks source link

Feature Request: Support "Missing" types on System.Reflection.MetadataLoadContext #96371

Open wasabii opened 8 months ago

wasabii commented 8 months ago

I don't really expect this request to be accepted, because it's a bit out odd. But I wanted to document the issue for posterity.

I am working on finally rewriting IKVM.Reflection around S.R.M. And, trying to consume as much existing shared code as possible. IKVM.Reflection's API surface mostly mirrors System.Reflection, and was one of the first libraries to do assembly generation from purely managed code without using System.Reflection.

As a consequence, we have a ton of compiler code that works against the System.Reflection API. Except, we substitute it out for IKVM.Reflection in our static compiler using #ifdef. The dynamic compiler continues to use System.Reflection directly.

One option is to just rewrite IKVM.Reflection's loading and emitting API surface against S.R.M and S.R.M.E.

The other is to investigate whether other existing System.Reflection-looking APIs are at a point where we can just use them instead, and deprecate IKVM.Reflection completely. No AssemblyBuilder.Save puts the Emit side on hold for now.

But, System.Reflection.MetadataLoadContext may be a viable operation for the assembly-reading side. Except for one feature that IKVM uses: an IsMissing and ContainsMissing property on Type, Method, Field, etc.

The point of the IsMissing property on a Member is to allow a compiler to generate code that deals with partially complete assemblies. For instance, a TypeSpec in Assembly A, being built against Assembly B, where Assembly B is missing one of the target TypeRefs, can return Type with IsMissing as true. So our compiler can know this information, and use it to generate metadata against an incomplete type model. We use this to preserve Java runtime semantics in statically compiled IL.

For instance, in the example above, Assembly A might have a method that inits an object of TypeB in Assembly B, where Assembly B was generated without that type (perhaps an older version of Assembly B). Java semantics would have this method call fail at runtime, but only when it hits the portion of code where TypeB is actually required. As such, we generate the IL body for the method in Assembly A to throw the appropriate exception at that point. We emit IL to throw MethodNotFoundException. Which preserves Java semantics.

With IKVM.Reflection, we get a IKVM.Reflection.Type instead for this TypeSpec. The TypeSpec is fully resolved. But, might contain ElementTypes or generic instances, that are themselves unable to be resolved. The compiler can know this by checking ContainsMissing, and make a decision to emit a method body that throws.

With S.R.MLC, of course, any attempt to get or access the System.Type that is ultimately derived from that TypeSpec, throws TypeLoadException, leaving us unable to examine the contents of the TypeSpec and make a decision.

So, it looks like S.R.MLC is not going to be a solution for us. Which leaves us with two options: rewriting IKVM.Reflection against S.R.M and S.R.M.E, largely leaving the type model in tact; or fork S.R.MLC and introduce the notion of a Missing type. I haven't yet figured out which option I'm going to take. I'd lean towards forking, if the effort to introduce IsMissing and periodically synchronize S.R.MLC is a net benefit over maintaining our own type model.

Anyways, just thought I'd throw this up here. It's at least, as I see it, a valid use case for people attempting to build more complex IL generation routines. The type-model (S.R.MLC, System.Type, etc) is important for compilers. And, as of now, there isn't a managed-only type-model that I can find that deals with this specific circumstance: except, I'd imagine, the one in Roslyn and ours. :)

ghost commented 8 months ago

Tagging subscribers to this area: @dotnet/area-system-reflection See info in area-owners.md if you want to be subscribed.

Issue Details
I don't really expect this request to be accepted, because it's a bit out odd. But I wanted to document the issue for posterity. I am working on finally rewriting IKVM.Reflection around S.R.M. And, trying to consume as much existing shared code as possible. IKVM.Reflection's API surface mostly mirrors System.Reflection, and was one of the first libraries to do assembly generation from purely managed code without using System.Reflection. As a consequence, we have a ton of compiler code that works against the System.Reflection API. Except, we substitute it out for IKVM.Reflection in our static compiler using #ifdef. The dynamic compiler continues to use System.Reflection directly. One option is to just rewrite IKVM.Reflection's loading and emitting API surface against S.R.M and S.R.M.E. The other is to investigate whether other existing System.Reflection-looking APIs are at a point where we can just use them instead, and deprecate IKVM.Reflection completely. No AssemblyBuilder.Save puts the Emit side on hold for now. But, System.Reflection.MetadataLoadContext may be a viable operation for the assembly-reading side. Except for one feature that IKVM uses: an IsMissing and ContainsMissing property on Type, Method, Field, etc. The point of the IsMissing property on a Member is to allow a compiler to generate code that deals with partially complete assemblies. For instance, a TypeSpec in Assembly A, being built against Assembly B, where Assembly B is missing one of the target TypeRefs, can return Type with IsMissing as true. So our compiler can know this information, and use it to generate metadata against an incomplete type model. We use this to preserve Java runtime semantics in statically compiled IL. For instance, in the example above, Assembly A might have a method that inits an object of TypeB in Assembly B, where Assembly B was generated without that type (perhaps an older version of Assembly B). Java semantics would have this method call fail at runtime, but only when it hits the portion of code where TypeB is actually required. As such, we generate the IL body for the method in Assembly A to throw the appropriate exception at that point. We emit IL to throw MethodNotFoundException. Which preserves Java semantics. With IKVM.Reflection, we get a IKVM.Reflection.Type instead for this TypeSpec. The TypeSpec is fully resolved. But, might contain ElementTypes or generic instances, that are themselves unable to be resolved. The compiler can know this by checking ContainsMissing, and make a decision to emit a method body that throws. With S.R.MLC, of course, any attempt to get or access the System.Type that is ultimately derived from that TypeSpec, throws TypeLoadException, leaving us unable to examine the contents of the TypeSpec and make a decision. So, it looks like S.R.MLC is not going to be a solution for us. Which leaves us with two options: rewriting IKVM.Reflection against S.R.M and S.R.M.E, largely leaving the type model in tact; or fork S.R.MLC and introduce the notion of a Missing type. I haven't yet figured out which option I'm going to take. I'd lean towards forking, if the effort to introduce IsMissing and periodically synchronize S.R.MLC is a net benefit over maintaining our own type model. Anyways, just thought I'd throw this up here. It's at least, as I see it, a valid use case for people attempting to build more complex IL generation routines.
Author: wasabii
Assignees: -
Labels: `area-System.Reflection`, `untriaged`
Milestone: -
steveharter commented 8 months ago

The other is to investigate whether other existing System.Reflection-looking APIs are at a point where we can just use them instead, and deprecate IKVM.Reflection completely. No AssemblyBuilder.Save puts the Emit side on hold for now.

Note than AssemblyBuilder.Save support is in progress. This started in 8.0 and continues into 9.0.

Extending AB.Save and S.R.MLC that addresses the missing member scenarios makes sense to me if it addresses your scenarios and enables a single solution for compiler writers and others. However, that will require community research and contributions since we don't have the bandwidth at this time.

ghost commented 8 months ago

This issue has been marked needs-author-action and may be missing some important information.

steveharter commented 7 months ago

@wasabii any thoughts on working together to add features to S.R.MLC and leveraging the AB.Save() work being added in 9.0 (it is functional now, but still a few features being worked on)?

wasabii commented 7 months ago

I'd be happy to. Problem is, I'm not sure whether this is even something S.R.MLC even wants to support.

IS4Code commented 6 months ago

I'd definitely appreciate improvements in this area, similarly in relation to #86923.