dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.04k stars 4.68k forks source link

Question: finding architecture using System.Reflection.Metadata #74040

Closed Forgind closed 2 years ago

Forgind commented 2 years ago

Using SRM was what @jkotas recommended in https://github.com/dotnet/runtime/issues/59061#issuecomment-922392861. I had trouble figuring out how to do that, exactly, and he told me to file a new issue. Can someone point me to an example of how to do that?

ghost commented 2 years ago

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata See info in area-owners.md if you want to be subscribed.

Issue Details
Using SRM was what @jkotas recommended in https://github.com/dotnet/runtime/issues/59061#issuecomment-922392861. I had trouble figuring out how to do that, exactly, and he told me to file a new issue. Can someone point me to an example of how to do that?
Author: Forgind
Assignees: -
Labels: `area-System.Reflection.Metadata`
Milestone: -
jkotas commented 2 years ago

This example shows how to create a System.Reflection.Metadata reader: https://docs.microsoft.com/en-us/dotnet/api/system.reflection.metadata.metadatareader?view=net-6.0#examples

Once you have the reader, PEHeaders.CoffHeader.Machine has the PE file architecture. For example, you can add Console.WriteLine(peReader.PEHeaders.CoffHeader.Machine) to print it.

What are you trying to do with this architecture? Assemblies in modern .NET are typically built as AnyCPU even when they are architecture or OS specific, and then packaged for specific RID to capture the fact that they are architecture or OS specific.

Assemblies are typically built for concrete architecture only for compatibility with .NET Framework.

ghost commented 2 years ago

This issue has been marked needs-author-action and may be missing some important information.

Forgind commented 2 years ago

PE isn't the layer I'm after. That's a windows concept; the code I'm replacing used AssemblyName.GetAssemblyName(\<dll>).ProcessorArchitecture, and that's a .NET concept. There are two ways we're currently using it:

  1. To figure out if something is MSIL or None (versus other). We use this in figuring out if a file matches a particular assembly name or if we should try to find another file that better matches what our users requested.
  2. If someone has an assembly specifically built for, say, x86, we need to know that our current process's architecture doesn't match then start a new process with the correct architecture to load and execute their code. We support ARM64, x86, and x64, at the moment, so I want a way to detect which an assembly is built for if it's specifically built for one of those.
jkotas commented 2 years ago

To figure out if something is MSIL or None (versus other).

You can do that by checking whether CorHeader is non-null.

If someone has an assembly specifically built for, say, x86, we need to know that our current process's architecture doesn't match then start a new process with the correct architecture to load and execute their code

AssemblyName.ProcessorArchitecture in .NET Framework was inferred from the PE file architecture. It is not first class .NET concept and that's why it was deprecated.

Forgind commented 2 years ago

AssemblyName.ProcessorArchitecture in .NET Framework was inferred from the PE file architecture. It is not first class .NET concept and that's why it was deprecated.

Can you link to where that was done? As a test case, I tried to figure out ProcessorArchitecture from CoffHeader, and it mostly worked, but there were cases where it didn't, and I don't know what the difference was for .dlls.

Forgind commented 2 years ago

For more context, we let the user pass in a string representing the architecture they care about. We parse it into a System.Reflection.ProcessorArchitecture, then try to compare it to AssemblyName.ProcessorArchitecture. If there's a way to map CoffHeader.Machine to AssemblyName.ProcessorArchitecture, that'd be great, but Machine has a lot more in its enum, and I don't know what maps to what.

jkotas commented 2 years ago

If there's a way to map CoffHeader.Machine to AssemblyName.ProcessorArchitecture,

There is not.

ProcessorArchitecture was implemented by stealing a few bits from AssemblyNameFlags. It severely limits how many different processor architectures can be represented by ProcessorArchitecture. It was one of the contributing reasons for obsoleting it. We are not adding new architectures supported by .NET to ProcessorArchitecture since we would quickly hit a dead end.

jkotas commented 2 years ago

Can you link to where that was done?

The code still exists here for backward compatibility: https://github.com/dotnet/runtime/blob/6d0bcc4cc7cf98e661c91d4f2abace2c5bd282a5/src/coreclr/System.Private.CoreLib/src/System/Reflection/AssemblyName.CoreCLR.cs#L113

Forgind commented 2 years ago

I'm still missing two pieces.

I wanted to go from file to ProcessorArchitecture, and I can get from file to Machine (which seems to be equivalent to the ImageFileMachine in the method you linked?) and PortableExecutableKinds to ProcessorArchitecture. How do I get a PortableExecutableKind?

Second, in the issue I'd originally commented on, you said I should use System.Reflection.Metadata. All I've seen so far is System.Reflection. Am I missing something?

jkotas commented 2 years ago

I wanted to go from file to ProcessorArchitecture

ProcessorArchitecture does not have value for Arm64. What are you going to do for Arm64? I do not think that the plan with using ProcessorArchitecture is going to work. You should switch over to a non-obsolete enum. The PE file enum Machine seems to be closest to what you are trying to achieve.

How do I get a PortableExecutableKind?

The logic to compute PortableExecutableKind is here: https://github.com/dotnet/runtime/blob/17154bd7b8f21d6d8d6fca71b89d7dcb705ec32b/src/coreclr/inc/pedecoder.inl#L1096-L1156 . Note that it is very Windows and .NET Framework specific. It does not make sense on non-Windows platforms and in .NET Core/5+.

AraHaan commented 2 years ago

Is there an .NET way to check the architecture of the .NET Host?

Example:

What happens in my program is that I do not know how to check the bitness the host in running in when it's AnyCPU and as such could crash when it selects the x64 plugin instead of the x86 one when in x86 host, or selects the x86 one when in x64 host which is not good at all. I could have it target specific x86 or x64 but then it will remove LargeAddressAware (for x86) which is also not good at all in the program (as it might need it set). I could set it manually if it was not for setting it up for the compiler to sign the code with an digital signature (modifying it afterwards invalidates the signature which is also not good as then people won't trust the assembly anymore when it is shipped with an invalid signature). Yes I sign my assemblies so that they know "It was not tampered with after it was compiled from CI on tag builds and deployed."

jkotas commented 2 years ago

Is there an .NET way to check the architecture of the .NET Host?

RuntimeInformation.ProcessArchitecture returns the architecture of .NET host for the current process.

AraHaan commented 2 years ago

Alright, I guess I could use that in my program to split plugins based on RID and require C++ plugins to output into an RID specific folder and then load plugins in this order:

And hopefully avoid said problems in my program then.

Forgind commented 2 years ago

RuntimeInformation.ProcessArchitecture returns an Architecture. It sounded like your recommended way to get the architecture of a file is to get Machine from the PEHeader. Is there a built-in comparison method?

AraHaan commented 2 years ago

I actually found an even better way of checking the architecture:

it's System.Runtime.InteropServices.RuntimeInformation.RuntimeIdentifier which then becomes the runtime identifier of the current running process of which can only be the ones supported by the .NET SDK and the runtime.

That makes code like this utterly a waste of time (yes I did it before the above was pointed out to me) https://gist.github.com/AraHaan/0fdcfc658a909034bd5409f580088896

jkotas commented 2 years ago

It sounded like your recommended way to get the architecture of a file is to get Machine from the PEHeader. Is there a built-in comparison method?

There is no built-in comparison method. Nothing that we have needs a comparison method like this.

As I have said in https://github.com/dotnet/runtime/issues/74040#issuecomment-1217483633 , the target architecture in modern .NET is identified by the RID that the assembly was packaged for. .NET SDK does not try to guess what the assembly target architecture may be by analyzing content of the binary.

MSDN-WhiteKnight commented 2 years ago

In addition to CoffHeader.Machine, CorHeader.Flags might be relavant to you if you care about .NET Framework compat. In my tests, assemblies compiled for .NET Framework seems to have machine type set to I386 even if they are platform-agnostic; but those that are really x86-specific have Requires32Bit corflag set.