Support for Apple M1 arm64/aarch64 chips

[Willena]

I've been using this docker container to cross-compile some libraries for Mac OS for quite a long time. It works great ! After a quick look to the content of the container, it seems that it contains some of the things needed to build for aarch64 / Apple M1 processor. Adding a new CROSS_TRIPLE case and setting -arch arm64 with an arm64 compatible GCC could do the work.

I don't know how difficult it would be to implement this new processor. Correct me if I'm wrong.

[ofek]

cc @moul @junaruga @aimxhaisse @lafin @QuentinPerez

Hello! Has there been any progress on this?

[junaruga]

Sorry I do not know any progress.

[ofek]

@Willena Could you try to open a pull request?

[kroggen]

I suspect that currently only clang compiles to Apple Silicon, requiring Xcode 12.2 on Mac.

gcc has a suspended ticket for its support, but there is a work-in-progress implementation here.

[mcandre]

The upstream crossbuild project has added code for M1 ARM (64) support.

https://github.com/tpoechtrager/osxcross/commit/261de1051107e47e34a0afc2aed75b8173bc6856

[Willena]

The only issue is that to target arm64e ( M1 chips if I'm correct ) we still need to build the llvm-compiler from scratch (not very difficult, but requires a lot of computing resources and it is a quite long process). I'm not very familiar with arm architecture, but if arm64 is compatible with arm64e then this becomes much easier.

[mcandre]

Confirmed, building the Apple Clang compiler is able to generate arm64[e] toolchains for osxcross. This is documented in the osxcross README.

https://github.com/tpoechtrager/osxcross#what-is-the-goal-of-osxcross

Example compilation of Apple Clang on Ubuntu:

https://github.com/mcandre/apple-clang/blob/master/Dockerfile

[yuzawa-san]

My use case was to build a universal2 binary on x86_64. I was able to get it working by doing the following:

• The old 10.10 SDK is too old and I needed to upgrade. Per https://github.com/tpoechtrager/osxcross/blob/1c23b94bad4efc78d66ae917fa173c0d27bd2545/wrapper/target.cpp#L527 and https://github.com/tpoechtrager/osxcross/blob/062922bbb81ac52787d8e53fa4af190acb552ec7/build.sh#L43 I'll need at least 11.0. I set osxcross_revision="062922bbb81ac52787d8e53fa4af190acb552ec7" to a recent revision (head of master) from today to get support for newer SDKs. I setdarwin_sdk_version="11.3",darwin_osx_version_min="10.9", darwin_version="20.4", and set darwin_sdk_url to a local version of SDK 11.3 extracted using the osxcross instructions.
• osxcross then failed to build, so I then upgraded to debian bullseye: FROM buildpack-deps:bullseye-curl which is currently underway in https://github.com/multiarch/crossbuild/pull/61 this is required since osxcross needs a newer cmake, only present in bullseye: https://github.com/tpoechtrager/osxcross/issues/235#issuecomment-675568010
• This I added set(CMAKE_OSX_ARCHITECTURES arm64 x86_64) to my project's CMakeLists.txt to build a universal2 binary.
• NOTE: I did not add/remove any CROSS_TRIPLES since the clang on x86_64 was able to cross compile just fine.

I can make a PR if you want, but I don't know where that original SDK came from (dropbox?). Perhaps we can add a copy step to the Dockerfile. Also it may become necessary to do more creative tagging as mentioned in https://github.com/multiarch/crossbuild/issues/48 as to support more diverse sets of build environments:

• stretch-catalina
• stretch-bigsur, stretch
• ~stretch-monterrey, stretch~ this is not a supported combination for reasons mentioned above
• bullseye-catalina, catalina
• bullseye-bigsur, bigsur
• bullseye-monterrey, monterrey, bullseye, latest

[andrewleech]

@yuzawa-san I believe the newer cmake can typically be installed via python, eg. apt-get install -y python3-pip; python3 -m pip install cmake which may remove the need to upgrade distribution for this?

[gotson]

I'm trying to make this work without cmake, as our current build chain uses make, i need to pass the path to the cross-compiler (i suppose clang from what i read above).

Anyone managed to do that ?

I can compile our library, and it seems to be recognized as arm64, but when i try to use it the process crash without much information. I'm probably doing something wrong.

[gotson]

Turns out it was a signature problem, as M1 macs always check the signature. You need at least the adhoc signature for a binary to run.

[andrewleech]

@gotson that all sounds quite promising, nice job figuring out the couple of extra steps!

So are you able to build a binary using a docker built from your fork/commit referenced above?

That ad-hoc signing, can that be done with the tools in the docker too?

[gotson]

@andrewleech yes so far it works, i still need to do a couple of checks to make sure everything is fine before merging this to the master branch.

We are building a dynamic library for multiple platforms, and after some digging i found out a few things:

• when building on a M1 mac with gcc -arch arm64 or clang, there is an adhoc signature. If you strip the binary, the signature is still there (probably some Apple magic).
• when building using crossbuild with clang, the adhoc signature is also there. However, when you strip there is a warning /usr/osxcross/bin/aarch64-apple-darwin20.4-strip: warning: changes being made to the file will invalidate the code signature in: /tmp/libsqlitejdbc.jnilib, and the signature is removed.
• when you run a binary that is not signed (adhoc signature is fine) on a M1 mac, Gatekeeper will send a SIGKILL straight away.
• On a Mac you can use the codesign binary to verify a signature: codesign --verify libsqlitejdbc.jnilib
• You can also use it to add an adhoc signature: codesign --sign - libsqlitejdbc.jnilib

While checking the issues for M1 mac in the osxcross repo, i stumbled upon this: https://github.com/tpoechtrager/osxcross/issues/355#issuecomment-1213001460

Given codesign is not available on Linux, i created my own Docker image with rcodesign (source). That way i can sign Mac binaries from Docker.

I use a fork of crossbuild following the super helpful comments of @yuzawa-san above.

It seems another tool (sigtool) could be automatically called by osxcross, but i didn't manage to get that working.

[andrewleech]

Thanks for those details! I can see from the osxcross issues some more details about sigtool:

• https://github.com/tpoechtrager/osxcross/issues/305#issuecomment-932831959
• https://github.com/thefloweringash/sigtool It certainly looks like one valid solution with the following caveat:

  Currently only supports embedded ad-hoc signatures for universal and thin 64-bit Mach-O files.

I'm guessing I could also just skip strip for a basic open-source executable :-D

[gotson]

I'm guessing I could also just skip strip for a basic open-source executable :-D

Not an option for us, we ship the library in the jar, and every kb counts! Stripping saves around 100-200kb.

Note that rcodesign doesn't have those limitations from what i checked, so you could even use it to sign with your AppleID, and it can also notarize your app.

[KhanMechAI]

any update on this?