macOS: cross-compile x86_64 + arm64

[tyrone-sudeium]

This PR amends the osx toolchain so it emits static libraries built for Apple Silicon (arm64), and for Rosetta / Legacy Intel (x86_64).

We can adopt these libraries into the Player at our leisure, but do note that this PR moves the resulting cmake prefix into a sub folder, so at a minimum all the other Jenkins scripts (and documentation, etc) will need to be updated to buildscripts/osx/x64_64.

WIP because I still get a weird failure in icu_cross where it can't find #include <typeinfo> but I've only had this happen when running the script cleanly, if I rebuild just the arm64 side after the failure it starts working...

I'm interested to see what the Jenkins agent does, since I'm currently testing this on a macOS 12 Monterey system, which is installed on the only Mac I've got access to, and it still has all sorts of weird bugs.

• [x] liblcf & Player tests
• [x] Test if Player runs in x64 systems
• [x] Test if Player runs on ARM systems
• [ ] Update Jenkins jobs

[Ghabry]

My Jenkins Agent is still on 10.15.4. Is that good enough for arm64? xD

(yeah have to update it... but this has to wait two more months, need some longer freetime for this)

[tyrone-sudeium]

For 10.15.4 you'll want to be running Xcode 12.2 to 12.4. Anything older doesn't support arm64 in the macOS SDK, anything newer requires macOS 11+.

[Ghabry]

FYI the Jenkins System is now on Monterey and has Xcode 13.3. (at least when "Expanding Xcode.xip" is finished, wow this takes hours o_O)

[Ghabry]

Rebased and the building of the libraries works.

Pending stuff added to initial comment

[tyrone-sudeium]

So it doesn't get lost in my terminal history / Discord history, here's a summary of the next steps:

We can just inch our way toward macos-arm64 support bit by bit. To start with:

• let's just merge this buildscripts PR, and then
• update the liblcf and player-macos jobs to simply do exactly what they do right now (build only x86_64) but updated for the new path.

After it's all working, the next step is making liblcf support it. There's a decision to be made here:

• do we merge both liblcf.a files to create a liblcf universal binary?
  • nice because we don't have to update player, tools, etc build jobs a second time
• have it work the same as toolchains where we move each arch into a subfolder
  • this makes it more consistent with toolchains and also android
  • i've tested this approach and demonstrated it working, and the examples below assume this is the option picked
• Easier now, but we'll probably need to migrate the Jenkins job to build it using cmake, using DCMAKE_OSX_ARCHITECTURES="x86_64;arm64". Unless you have some way of making autotools do that?

After liblcf is sorted, we fix Player, tools, anything else we want to support universal binaries, as per your comment below.

Last but not least, we update the website to make the new universal binary available to users 😄

[Ghabry]

Makes me wonder if there are already Cmake powered solutions available that use lipo. Integrates better into the buildsystem

[Ghabry]

Can you make these changes and test if they work on your system? At least it builds and links Player fine for me xD

By using --disable-assembly in SDL2 (otherwise SDL_cpuinfo.h leaks in and breaks the build) it is possible to merge the libraries after building them.

Don't think this will have a significant negative speed impact as we almost do not use SDL2 and the new macOS hardware is extremely fast. :P

Everything else still uses intrinsics (noteworthy is pixman here).

Changes (note that the diff has incorrect whitespace [space, instead of tab], I cannot easily grab files from macOS, please fix them manually). Maybe just apply it manually... is short

diff --git a/macos/0_build_everything.sh b/macos/0_build_everything.sh
index 14c13b4..661f7aa 100755
--- a/macos/0_build_everything.sh
+++ b/macos/0_build_everything.sh
@@ -7,4 +7,5 @@ fi

 ./1_download_library.sh \
        && ./2_build_toolchain.sh \
-       && ./3_cleanup.sh
+       && ./3_build_universal.sh \
+       && ./4_cleanup.sh
diff --git a/macos/2_build_toolchain.sh b/macos/2_build_toolchain.sh
index 98d229d..eea5ee9 100755
--- a/macos/2_build_toolchain.sh
+++ b/macos/2_build_toolchain.sh
@@ -83,7 +83,7 @@ function build() {
        install_lib_cmake $FMT_DIR $FMT_ARGS
        install_lib_icu_cross
        install_lib_liblcf
-       install_lib $SDL2_DIR $SDL2_ARGS
+       install_lib $SDL2_DIR $SDL2_ARGS --disable-assembly
 }

 export MAKEFLAGS="-j${nproc:-2}"

3_build_universal.sh (dont forget chmod +x before commit):

#!/bin/bash

if [ "$(uname)" != "Darwin" ]; then
        echo "This buildscript requires macOS!"
        exit 1
fi

# abort on error
set -e

echo "Creating universal libraries"

# prepare
rm -rf universal
mkdir -p universal

# copy files needed by CMake, doesn't matter if x64 or arm
cp -R x86_64/bin x86_64/include x86_64/lib universal

# merge the libraries with lipo
for x64_file in $(find x86_64/lib -type f -name "*.a")
do
    filename=$(basename $x64_file)
    arm_file="arm64/lib/$filename"
    uni_file="universal/lib/$filename"

    echo "Merging $filename"

    lipo -create "$x64_file" "$arm_file" -output "$uni_file"
done

This reduces the build line to:

(important is CMAKE_OSX_ARCHITECTURES="x86_64;arm64")

cmake . -Bmacosbuild -GNinja -DCMAKE_PREFIX_PATH='/Users/macos/jenkins/workspace/player-macos/../macos-toolchain/macos/universal;/Users/macos/jenkins/workspace/player-macos/../liblcf-macos' -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_OSX_DEPLOYMENT_TARGET=10.9 -DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=/Users/macos/jenkins/workspace/player-macos/universal
cmake --build macosbuild --target check
cmake --build macosbuild --target install

> lipo -archs macosbuild/EasyRPG\ Player.app/Contents/MacOS/EasyRPG\ Player
x86_64 arm64