Compilation failing for calli with fastcall calling convention

[dadavadd]

There are no errors with Cdecl, but when you select fastcall, the following error appears.

My Method:

How can I fix this error so that I can compile to fastcall?

[dadavadd]

NET 8.0

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas See info in area-owners.md if you want to be subscribed.

Issue Details

---

 There are no errors with Cdecl, but when you select fastcall, the following error appears. My Method:  How can I fix this error so that I can compile to fastcall?

Author:	dadavadd
Assignees:	-
Labels:	`untriaged`, `area-NativeAOT-coreclr`
Milestone:	-

[filipnavara]

NativeAOT doesn’t support x86 platform which is the only platform where fastcall calling convention is applicable.

(Obviously the compiler should not fail either and should ignore the calling convention.)

[MichalStrehovsky]

This is failing because RyuJIT asks canGetCookieForPInvokeCalliSig over JitInterface and it's not implemented for native AOT.

https://github.com/dotnet/runtime/blob/bee4602cd5974c9abb2d83cc58d5342b75b4cdbe/src/coreclr/jit/importercalls.cpp#L670-L716

Dropping the fastcall (and leaving it at the default unmanaged calling convention) fixes this.

Should RyuJIT be asking this and making decisions about a fastcall when we target x86-64? Is fastcall doing anything at all outside x86? I guess we could implement it, but is this meaningful?

Cc @dotnet/jit-contrib

[jkotas]

Native AOT does not have calli cookies.

Unmanaged calli should be either converted to regular call of IL stub by convertPInvokeCalliToCall or inlined by the JIT without needing a cookie.

The inlined unmanaged calli should take this path: https://github.com/dotnet/runtime/blob/bee4602cd5974c9abb2d83cc58d5342b75b4cdbe/src/coreclr/jit/importercalls.cpp#L5625-L5629 . The problem is that we have taken an early out a few lines earlier and that makes us wonder into the calli cookie land.

Regular CoreCLR deals with the problem by duplicating the error condition for supported calling conventions here: https://github.com/dotnet/runtime/blob/bee4602cd5974c9abb2d83cc58d5342b75b4cdbe/src/coreclr/vm/dllimport.cpp#L4264-L4270 . The simplest fix would be to follow to suit, duplicate this error condition in the aot compiler (sigh), and generate a throwing stub in convertPInvokeCalliToCall for unsupported calling conventions.

[jkotas]

Is fastcall doing anything at all outside x86?

fastcall was never implemented on x86, and we throw for it on non-x86 too just because.