dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.37k stars 4.75k forks source link

Ability to generate header file from exports #100747

Open am11 opened 7 months ago

am11 commented 7 months ago

Methods decorated with [UnmanagedCallersOnly(EntryPoint="export-name")] end up as exported symbols in published binary. Since ILC has all the required info related to method signatures and beyond, it can generate a plain C header file {outputFileName}.h with exported APIs accurately for user convenience.

The main work it entails is C# to C type mapping (int->int32_t etc. with inttypes.h, stdbool.h includes), generating enums and generating complex object graphs representation in terms of structs. This feature can be enabled behind an optional project property <GenerateCHeaderFile>true. Further customization may not required and ilc can be opinionated about the style choice (e.g. all definitions stuffed in one header file).

dotnet-policy-service[bot] commented 7 months ago

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas See info in area-owners.md if you want to be subscribed.

jkotas commented 7 months ago

This applies to interop in all runtime form factors, it is not specific to native AOT. You can use https://github.com/AaronRobinsonMSFT/DNNE to generate this header file today.

Further customization may not required

Customization is typically required for anything that is a bit more complex. Check the customizations implemented in DNNE https://github.com/AaronRobinsonMSFT/DNNE?tab=readme-ov-file#native-code-customization .

am11 commented 7 months ago

This could be a subset of DNNE (only for exports) implemented directly in runtime repo.

It may have higher demand in AOT realm (e.g. https://github.com/dotnet/runtime/pull/100623/files#diff-3dad50f2b179b16bb7f3856dba5f614fea3e838c65e710885f0fc17f989ac925); but if it can be done agnostic of form-factor, that would be better. Alternatively, since ILCompiler already looks for exports for other functionalities, it can be extended for this feature without expanding feature's scope.

Method / Type infos of exported APIs and their arguments to C will cover majority of the use-cases, and provide something functional for the rest without customization. e.g. overlapping [FieldOffset] decorated members that are part of exports graph to end up in C union. In best case, user can take the ilc generated shared lib and header file, and use it in their project without modifications. Otherwise, they can skip the header generation if they are using something else (existing header).

AaronRobinsonMSFT commented 7 months ago

@am11 I agree this does have utility and would address the generation of "boilerplate". Although I'm not sure it is at a place where we should be providing it in-box. The vast majority of signatures, since they are unmanaged, are trivial to author and the flow of these unmanaged artifacts in the managed build system creates a burden that I have trouble justifying. I would also say that C header files might be the desire now, but this feature is ripe for complex feature creep. What about definitions in Rust? Swift? Should the header be pure C99 or fully C++ compliant? Where do these artifacts go in a NuGet package? Are they included in a publish action? The introduction of extra unmanaged concepts expand the matrix greatly and concerns me. I've had a bunch of nits in DNNE that give me pause about making this feature a first-class in-box feature.

am11 commented 7 months ago

What about definitions in Rust? Swift?

If consumer is using the native AOT generated objects in project written in language other than C/C++, they would still be able to use third-party tool to convert the generated C header file to the target language's equivalent representation. Today, they are collecting the entrypoints definitions and writing it manually, which is more laborious than having a flat C header file to work with.

Should the header be pure C99 or fully C++ compliant?

It can support both simultaneously, e.g. MyProject.API.h:

// This file is generated by .NET Native IL Compiler.

#include <inttypes.h>
#include <stdbool.h>

#ifdef __cplusplus
extern "C"
{
#endif

// consts
// enums
// structs
// methods

#ifdef __cplusplus
}
#endif

Where do these artifacts go in a NuGet package? Are they included in a publish action?

I think it can use the same plan as symbol files (.dbg, .pdb, .dSYM); only generated with dotnet publish and end up in PublishDir (next to the binary).

AaronRobinsonMSFT commented 7 months ago

which is more laborious than having a flat C header file to work with.

Agree. The push back I have, at least at the moment, is (a) about understanding how common the native AOT library scenario is going to be and then (b) the additional overhead/management for producing and handling unmanaged assets. If we get signal, I am currently working with the VS team to help get data for this very question, that the library is a non-trivial portion of native AOT scenarios then we can move onto (b) and understand how to manage those assets.

For now though, we need an indication that (a) is worth the investment in designing a solution for all that (b) is presently and will likely involve into. Per usual, getting community traction on this issue will also help provide signal with respect to (a).