Compiling for x86_64 on M1 Mac

[jamsoft]

I've picked up a project using this to produce a native dylib.

The windows version is all working perfectly. The issue starts when I try to compile it for x64 mac on an M1 machine.

The rest of my solution is cross compiling fine but I just can't seem to get this project compiling. The errors I'm getting are:

  dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.11/libhostpolicy.dylib, 0x0001): tried: '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.11/libhostpolicy.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

And

  DNNE.targets(76, 5): [MSB3073] The command "dotnet  "/Users/<>/.nuget/packages/dnne/1.0.32/build/../tools/dnne-gen.dll" obj/Debug/net6.0/osx-x64/My.dll  -o obj/Debug/net6.0/osx-x64/dnne/My.g.c" exited with code 130.

In my other builds a use scripts executed like arch -x86_64 ./myscript in order to execute as x64 not arm64. I also have switched my terminal app to always use i386 and even executing the commands this way fails.

Also a bit confusing why Microsoft.NETCore.App/5.0.11/ is mentioned in a .NET 6.0.3 application?

Any ideas would be great.

[jamsoft]

I've just noticed the note in the docs ...

86, x86_64, ARM64 compilation supported. The Visual Studio package containing the desired compiler architecture must have been installed.

So in my main project I have explicitly referenced the various ILCompiler packages, i.e.:

<PackageReference Include="runtime.osx-x64.Microsoft.DotNet.ILCompiler" Version="7.0.0-preview.5.22301.12" />

But I'm not sure if these are relevant to DNNE. Any pointers would be great.

[jamsoft]

So I've just tidied up my installed SDKs so I only have 6x SDKs and Runtimes installed. Now when compiling I'm getting this error:

no such file or directory: '/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Host.osx-x64/6.0.5/runtimes/osx-x64/native/libnethost.a'

Hmm ...

[jamsoft]

Just discovered that this directory is actually missing ...

no such file or directory: '/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Host.osx-x64/6.0.5'

I'm not sure how to get this installed. Is this a missing 'targetting pack' install?

[AaronRobinsonMSFT]

This is a cross-targeting issue that is going to need some investigation on my end. I'm not sure how or even if we pull down the x64 host assets on an M1.

@vitek-karas or @elinor-fung Any thoughts here?

[vitek-karas]

I don't know how DNNE does this, but the host packages should be "download on demand". The package exists on NuGet: https://www.nuget.org/packages/Microsoft.NETCore.App.Host.osx-x64/6.0.5.

dotnet build when building the apphost uses this package. It will get it locally from the packs directory if available, and download from NuGet if not.

On Windows the installer will populate the packs folder with all Windows architectures (x86, x64, arm64), but it probably doesn't do that on Mac. For SDK it should not matter, it should just download it, but DNNE probably doesn't work that way.

[AaronRobinsonMSFT]

I don't know how DNNE does this, but the host packages should be "download on demand". The package exists on NuGet: https://www.nuget.org/packages/Microsoft.NETCore.App.Host.osx-x64/6.0.5.

DNNE relies on the SDK to do everything, it does nothing special. I think I may have an idea now though.

@jamsoft As @vitek-karas mentioned, the SDK should be pulling this down for you. However, since you are cross targeting you may need to tell the SDK which platform you are targeting. Can you try defining the target RID for your library? This will let the SDK know what native components to pull down. This can also be an issue on Windows when trying to target x86 instead of x86_64. Some notes about this are found in the repo readme.md under step (3) in the following section.

[jamsoft]

Hi, it's already setting osx-x64 in the pertinent places.

I did a massive clean up today on my mac. Uninstalled all the old SDKs and runtimes. So I only had 6.x installed. I re-ran both x64 and arm64 SDK installers, just to see what the effect would be.

Still no 6.0.5 contents in the pack directory.

BUT.

It started compiling ...

I hit the dreaded dotnet is not a valid command situation and had to fix the symlink to repair it.

So that's some progress but my gitlab runner on the same box is having odd x64/arm64 issues. Even with arch -x86-64 ./stript commands.

the REALLY odd thing is I can watch that build fail in the GitLab UI with missing arm64 log messages. Re-execute it, and it passes. Odd.

This x86/x64/arm business is a tad rocky!

[AaronRobinsonMSFT]

@jamsoft Please let me know if you are still blocked. I don't think there is anything left on the DNNE side of things to address here.