Dogfooding instructions for .NET 8 NativeAOT on macOS x64 via Rosetta

christianscheuer commented 1 year ago

We're extremely happy that NativeAOT on Mac x64/arm64 should now work in .NET 8 daily builds. As one of the early proponents of CoreRT, we're looking forward to help beta test this in a large real world application.

However, I'm running into issues with using the .NET 8 daily builds of the SDK with ILC. We need to build macOS universal binaries (eventually) so all of our builds of x64 are done via Rosetta 2 on an Apple Silicon device.

Here's the repro: https://github.com/christianscheuer/nativeaot-net8

Steps:

I installed macOS x64 SDK version 8.0.100-alpha.1.22605.4 from github.com/dotnet/installer
All commands are run from a Rosetta Terminal.
The SDK host is x64, the SDK is x64.
Run ./build.sh which essentially just ensures we're in Rosetta, then runs dotnet publish -c Release -r osx-x64

Expected result:

It builds

Actual result:

MSBuild version 17.5.0-preview-22564-01+d14b74d6f for .NET
  Determining projects to restore...
  Restored /Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj (in 327 ms).
/usr/local/share/dotnet/x64/sdk/8.0.100-alpha.1.22605.4/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(275,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj]
  nativeaot-net8 -> /Users/chr/d/d2/nativeaot-net8/bin/Release/net8.0/osx-x64/nativeaot-net8.dll
/Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.DotNet.ILCompiler.SingleEntry.targets(51,5): warning : Delete explicit 'Microsoft.DotNet.ILCompiler' package reference in your project file. Explicit 'Microsoft.DotNet.ILCompiler' package reference can run into version errors. [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj]
/Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.NETCore.Native.Publish.targets(70,5): error : Add a PackageReference for 'runtime.osx-arm64.Microsoft.DotNet.ILCompiler' to allow cross-compilation for x64 [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj]

What's weird is that it claims it can't do cross-compilation, which I'm not actually doing. This is the x64 SDK running in an x64 terminal trying to build an x64 release. Furthermore, it requests the arm64 PackageReference. Adding that reference changes nothing, it reports the same error. It also complains about an explicit package reference that I'm not seeing.

I tried following the instructions on dotnet/installer to add the Nuget.config, but that ends up in yet other errors.

I shared the output of dotnet --info below.

Is there anything I should do differently to consume and test .NET 8 NativeAOT for macOS x64? Are there dogfooding instructions for this scenario somewhere that I haven't located?

`dotnet --info` output:

``` .NET SDK: Version: 8.0.100-alpha.1.22605.4 Commit: af14110139 Runtime Environment: OS Name: Mac OS X OS Version: 12.5 OS Platform: Darwin RID: osx.12-x64 Base Path: /usr/local/share/dotnet/x64/sdk/8.0.100-alpha.1.22605.4/ Host: Version: 8.0.0-alpha.1.22559.2 Architecture: x64 Commit: dd7fdb723d .NET SDKs installed: 5.0.408 [/usr/local/share/dotnet/x64/sdk] 6.0.403 [/usr/local/share/dotnet/x64/sdk] 8.0.100-alpha.1.22605.4 [/usr/local/share/dotnet/x64/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 5.0.17 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.11 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.0-alpha.1.22558.5 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 5.0.17 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.11 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.0-alpha.1.22557.12 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.0-alpha.1.22559.2 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Other architectures found: arm64 [/usr/local/share/dotnet] registered at [/etc/dotnet/install_location_arm64] Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```

cc @jkotas

ghost commented 1 year ago

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas See info in area-owners.md if you want to be subscribed.

Issue Details

We're extremely happy that NativeAOT on Mac x64/arm64 should now work in .NET 8 daily builds. As one of the early proponents of CoreRT, we're looking forward to help beta test this in a large real world application. However, I'm running into issues with using the .NET 8 daily builds of the SDK with ILC. We need to build macOS universal binaries (eventually) so all of our builds of x64 are done via Rosetta 2 on an Apple Silicon device. Here's the repro: https://github.com/christianscheuer/nativeaot-net8 Steps: - I installed macOS x64 SDK version 8.0.100-alpha.1.22605.4 from github.com/dotnet/installer - All commands are run from a Rosetta Terminal. - The SDK host is x64, the SDK is x64. - Run `./build.sh` which essentially just ensures we're in Rosetta, then runs `dotnet publish -c Release -r osx-x64` Expected result: - It builds Actual result: ``` MSBuild version 17.5.0-preview-22564-01+d14b74d6f for .NET Determining projects to restore... Restored /Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj (in 327 ms). /usr/local/share/dotnet/x64/sdk/8.0.100-alpha.1.22605.4/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(275,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj] nativeaot-net8 -> /Users/chr/d/d2/nativeaot-net8/bin/Release/net8.0/osx-x64/nativeaot-net8.dll /Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.DotNet.ILCompiler.SingleEntry.targets(51,5): warning : Delete explicit 'Microsoft.DotNet.ILCompiler' package reference in your project file. Explicit 'Microsoft.DotNet.ILCompiler' package reference can run into version errors. [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj] /Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.NETCore.Native.Publish.targets(70,5): error : Add a PackageReference for 'runtime.osx-arm64.Microsoft.DotNet.ILCompiler' to allow cross-compilation for x64 [/Users/chr/d/d2/nativeaot-net8/nativeaot-net8.csproj] ``` What's weird is that it claims it can't do cross-compilation, which I'm not actually doing. This is the x64 SDK running in an x64 terminal trying to build an x64 release. Furthermore, it requests the arm64 PackageReference. Adding that reference changes nothing, it reports the same error. It also complains about an explicit package reference that I'm not seeing. I tried following the instructions on dotnet/installer to add the Nuget.config, but that ends up in yet other errors. I shared the output of `dotnet --info` below. Is there anything I should do differently to consume and test .NET 8 NativeAOT for macOS x64? Are there dogfooding instructions for this scenario somewhere that I haven't located?

`dotnet --info` output:

``` .NET SDK: Version: 8.0.100-alpha.1.22605.4 Commit: af14110139 Runtime Environment: OS Name: Mac OS X OS Version: 12.5 OS Platform: Darwin RID: osx.12-x64 Base Path: /usr/local/share/dotnet/x64/sdk/8.0.100-alpha.1.22605.4/ Host: Version: 8.0.0-alpha.1.22559.2 Architecture: x64 Commit: dd7fdb723d .NET SDKs installed: 5.0.408 [/usr/local/share/dotnet/x64/sdk] 6.0.403 [/usr/local/share/dotnet/x64/sdk] 8.0.100-alpha.1.22605.4 [/usr/local/share/dotnet/x64/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 5.0.17 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.11 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.0-alpha.1.22558.5 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 5.0.17 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.11 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.0-alpha.1.22557.12 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.0-alpha.1.22559.2 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App] Other architectures found: arm64 [/usr/local/share/dotnet] registered at [/etc/dotnet/install_location_arm64] Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```

cc @jkotas

Author:	christianscheuer
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

jkotas commented 1 year ago

This is the x64 SDK running in an x64 terminal trying to build an x64 release.

The native AOT compiler defaults to running natively: https://github.com/dotnet/runtime/blob/c8f9f2949a243c411bd05278efab3f5a857cd0c1/src/coreclr/nativeaot/BuildIntegration/Microsoft.DotNet.ILCompiler.SingleEntry.targets#L20

You should be able to override this by setting IlcHostArch to x64. It will force the compiler to run as x64 emulated by Rosetta.

agocke commented 1 year ago

I believe the warning you're seeing is this bug: https://github.com/dotnet/runtime/issues/78647. It's an artifact of how we're downloading the package now. The warning shouldn't be appearing.

christianscheuer commented 1 year ago

Thanks, both. @jkotas I can confirm setting IlcHostArch to x64 worked. Is that the right default thing to do though (IlcHostArch using native arch instead of parent process arch)? I imagine this would be confusing to anybody else who builds x64 on Apple Silicon (which would be anybody who needs to build universal binaries) - but maybe this would be fixed instead by making cross compilation "just work"?

The assumption we have is that if the shell is x64, every process under it should run in x64 via Rosetta. That's how all of our other build tools do it (node, yarn, electron, Makefiles, CMake, clang, etc.) and it makes it very easy to target multiple archs on an Apple Silicon machine and never worry about cross compilation because every process just inherits the arch from its parent.

jkotas commented 1 year ago

I agree with you that it would make more sense to use the SDK architecture as the host architecture for the IL compiler. PublishReadyToRun compilation works like that. There is no reason for PublishAot to be different.

agocke commented 1 year ago

Hmm, personally it sounds like the process architecture is the right default, no? I believe the x64 terminal config in MacOS works by changing the process arch, but not the machine arch. If the publish takes the current process arch then you could use the -arm64 SDK to easily compile for x64 by just running it in the x64 process.

am11 commented 1 year ago

I tried following the instructions on dotnet/installer to add the Nuget.config,

Note that this file's name is case sensitive on Unix. Either all lower: nuget.config or exactly this casing: NuGet.Config.

I simply added an entry in the existing global config located on macOS at: ~/.nuget/NuGet/NuGet.Config

    <add key="dotnet8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json" />

(after the existing line <add key="nuget.org"..)

With that in place, the cross-compilation works as well:

% dotnet8 new console -n helloworldapp          

# workaround for https://github.com/dotnet/installer/pull/14991
# (templates are currently generating net7.0 projects by default until that PR is merged)
% sed -i '' 's/net7/net8/' helloworldapp/helloworldapp.csproj 

% dotnet8 publish helloworldapp -a x64 -o helloworldapp/dist -p:PublishAot=true

% file helloworldapp/dist/helloworldapp                                        
helloworldapp/dist/helloworldapp: Mach-O 64-bit executable x86_64

dotnet8 is an alias defined in ~/.zprofile:

alias dotnet8=~/.dotnet8/dotnet

This is a refresh script ~/update-daily-build.sh (which I run once a day):

#!/usr/bin/env bash

SECONDS=0

DEST="$HOME/.dotnet8"

# recreate destination directory
rm -rf "$DEST"
mkdir "$DEST"

# download and install
curl -sSL https://aka.ms/dotnet/8.0.1xx/daily/dotnet-sdk-osx-arm64.tar.gz | tar xzf - -C "$DEST"

# refresh caches
"$DEST"/dotnet nuget locals all --clear

"$DEST"/dotnet --info

duration=$SECONDS
echo "$(($duration / 60)) minutes and $(($duration % 60)) seconds elapsed."

vitek-karas commented 1 year ago

If the publish takes the current process arch then you could use the -arm64 SDK to easily compile for x64 by just running it in the x64 process.

I think we should run everything with the same architecture as the SDK which started it, unless it's an explicit choice to do otherwise. Having a random mix is just confusing, and honestly unexpected. Not counting that there's no guarantee that there will be an arm64 runtime available on the machine, so this could even fail to run.

christianscheuer commented 1 year ago

Thanks so much, everybody.

@am11 these instructions on consuming daily builds are extremely valuable, thank you for sharing. So much better not to pollute the regular dotnet installation with these daily builds but have more control. The nuget package clearing is neat too.

I got the x64 build working as cross compilation now with @am11's instructions now using the arm64 SDK, and it works just fine using either x64 or arm64 terminal.

I can also target arm64 using the arm64 SDK using those instructions, when in an arm64 terminal. Curiously, if I am in a Rosetta terminal and invoke the arm64 SDK, instead of getting a nice error saying I shouldn't do that, it fails with linker errors.

To repro that, follow @am11's instructions to install dotnet8.

Open Rosetta Terminal (or type arch -x86_64 zsh to open a sub shell in x64 mode).
Follow @am11's repro instructions, then run this at the publish step: dotnet8 publish helloworldapp -a arm64 -o helloworldapp/distarm -p:PublishAot=true

dotnet8 publish output

``` MSBuild version 17.5.0-preview-22564-01+d14b74d6f for .NET Determining projects to restore... Restored /Users/chr/d/d2/helloworldapp/helloworldapp.csproj (in 89 ms). /Users/chr/.dotnet8/sdk/8.0.100-alpha.1.22605.6/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(275,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/chr/d/d2/helloworldapp/helloworldapp.csproj] helloworldapp -> /Users/chr/d/d2/helloworldapp/bin/Debug/net8.0/osx-arm64/helloworldapp.dll /Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.DotNet.ILCompiler.SingleEntry.targets(51,5): warning : Delete explicit 'Microsoft.DotNet.ILCompiler' package reference in your project file. Explicit 'Microsoft.DotNet.ILCompiler' package reference can run into version errors. [/Users/chr/d/d2/helloworldapp/helloworldapp.csproj] Generating native code ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Net.Security.Native.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libbootstrapper.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libstdc++compat.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file obj/Debug/net8.0/osx-arm64/native/helloworldapp.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Security.Cryptography.Native.Apple.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libRuntime.WorkstationGC.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Security.Cryptography.Native.OpenSsl.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Globalization.Native.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Native.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 ld: warning: ignoring file /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.IO.Compression.Native.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 Undefined symbols for architecture x86_64: "_main", referenced from: implicit entry/start for main executable ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) /Users/chr/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/build/Microsoft.NETCore.Native.targets(346,5): error MSB3073: The command ""clang" "obj/Debug/net8.0/osx-arm64/native/helloworldapp.o" -o "bin/Debug/net8.0/osx-arm64/native/helloworldapp" /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libbootstrapper.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libRuntime.WorkstationGC.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/sdk/libstdc++compat.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Native.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Globalization.Native.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.IO.Compression.Native.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Net.Security.Native.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Security.Cryptography.Native.Apple.a /Users/chr/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-alpha.1.22559.2/framework/libSystem.Security.Cryptography.Native.OpenSsl.a -g -Wl,-rpath,'@executable_path' -ldl -lm -lobjc -lswiftCore -lswiftFoundation -lz -licucore -L/usr/lib/swift -framework CoreFoundation -framework CryptoKit -framework Foundation -framework Security -framework GSS" exited with code 1. [/Users/chr/d/d2/helloworldapp/helloworldapp.csproj] ```

agocke commented 1 year ago

If the publish takes the current process arch then you could use the -arm64 SDK to easily compile for x64 by just running it in the x64 process.

I think we should run everything with the same architecture as the SDK which started it, unless it's an explicit choice to do otherwise. Having a random mix is just confusing, and honestly unexpected. Not counting that there's no guarantee that there will be an arm64 runtime available on the machine, so this could even fail to run.

I think @christianscheuer summed up my understanding of the Apple approach to these things, and I agree this is the way it seems to work:

The assumption we have is that if the shell is x64, every process under it should run in x64 via Rosetta. That's how all of our other build tools do it (node, yarn, electron, Makefiles, CMake, clang, etc.) and it makes it very easy to target multiple archs on an Apple Silicon machine and never worry about cross compilation because every process just inherits the arch from its parent.

In this case we would target the arch of the process. For Native AOT at least, the concept of not having an x64 runtime doesn't matter -- the binary will have the correct architecture by definition.

agocke commented 1 year ago

Actually, when thinking about it I'm not sure everything works the way just described. In particular, does running in an x64 environment always run the x64 version of those tools, or do those tools just assume the architecture of the machine by the process they're running in?

This is something to investigate. Is clang running as an x64 process or is clang targeting x64? If it's running as an x64 process that does simplify things -- this would basically be a native targeting scenario, not a cross-targeting scenario.

That said, there is some inefficiency here, as in theory there could be performance improvements to actually doing cross-compilation using the arm64 SDK.

christianscheuer commented 1 year ago

Thank you - that's exactly the point I was trying to make, @agocke. Just tested - I can confirm clang runs as x64 process when invoked from x64 shell. It's a fat binary:

% file /usr/bin/clang
/usr/bin/clang: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64
- Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e
- Mach-O 64-bit executable arm64e]
/usr/bin/clang (for architecture x86_64):   Mach-O 64-bit executable x86_64
/usr/bin/clang (for architecture arm64e):   Mach-O 64-bit executable arm64e

And yes, I agree it would be possible to get perf optimizations by ignoring that and always run arm64 if available and make cross compilation work transparently to the end user "as if" it had been x64 all along. I would separate that out though as a potential performance optimization and get it to work in the usual Apple way first (everything inherits process arch).

Another point would be to consider adding a RID osx-universal or some other way to produce universal binaries. We have sufficient tooling in our own build system that it wouldn't be a big deal for us to create the fat binary ourselves - but I can think of many use cases where directly producing the fat binary would be a great addition to the SDK.

agocke commented 1 year ago

And yes, I agree it would be possible to get perf optimizations by ignoring that and always run arm64 if available and make cross compilation work transparently to the end user "as if" it had been x64 all along. I would separate that out though as a potential performance optimization and get it to work in the usual Apple way first (everything inherits process arch).

Yeah, only problem is that this is somewhat of an advanced scenario in .NET. We don't have fat binaries for the runtime so you have to install each architecture manually, and the entrypoints for each are different, so if you decide you want to execute the x64 version of the SDK you have to pick the x64 dotnet file manually.

christianscheuer commented 1 year ago

Absolutely, I didn't want to overcomplicate things. We can easily combine the binaries to a fat binary ourselves. Just wanted to be sure people were aware of the use case in the design choices.

The issue here was though that although I was using the x64 SDK, it would still run ILC in arm64 mode, which was what really caused the issue. If ILC just follows the process arch or SDK arch instead of the machine's native arch, then there shouldn't be any issues.

christianscheuer commented 1 year ago

We got our ~444K LoC app building and running successfully now on .NET 8 previews, through just about a half day's worth of troubleshooting and a few rd.xml patches – hooray! Great job to everybody on making this work.

In terms of libraries maintained by Microsoft that don't support trimming yet, I had to add a few rd.xml entries. Does it make sense to report issues in those libraries (AzureAD org) so that this can be tracked? I'm aware rd.xml is not a supported format, so would also love to know if there are other ways it's possible in code to ensure all metadata and methods are produced for a specific type (including specialisations). I tried various tricks like constructing an instance of a type, but this didn't seem to be enough to ensure all methods were created. Rd.xml is fine for now, but I noticed @MichalStrehovsky's PR to make it clear this is not a stable or supported format.

agocke commented 1 year ago

Yup it would be great to know what problems you’re running into

dotnet / runtime

Dogfooding instructions for .NET 8 NativeAOT on macOS x64 via Rosetta #79253