Apple Silicon (arm64) support

[sfuerte]

Symptom

getting a non-descriptive error: JRELoadError message on Apple M1

Problem

with some extra debugging, it clearly shows the failure is due to arch mismatch:

Searching for a JRE.
Required JVM must be at least ver. 7.
Searching for a Java 7
No matching JRE found.
Found a Java 11.jdk JDK
Looks like major version 11
JDK version qualifies
Java Runtime Dylib Path: '/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib'
Launchpath: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib
error: dlopen(/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib, 1): no suitable image found.  Did find:
    /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib: mach-o, but wrong architecture
    /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/MacOS/libjli.dylib: mach-o, but wrong architecture

> lipo -info /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/MacOS/libjli.dylib
Non-fat file: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/MacOS/libjli.dylib is architecture: arm64

But the current build is locked and forced to x86_64 only:

> lipo -detailed_info appbundler/bin/classes/com/oracle/appbundler/JavaAppLauncher
input file .../JavaAppLauncher is not a fat file
Non-fat file: .../JavaAppLauncher is architecture: x86_64

Solution

requires a bit more nicer one but as a proof of concept, this one worked for me:

diff --git a/build.properties b/build.properties
index aa28151..6963d12 100644
--- a/build.properties
+++ b/build.properties
@@ -25,4 +25,5 @@ version=1.0ea
 arch64=-arch x86_64
 arch32=-arch i386
 archppc=-arch ppc
+archarm64=-arch arm64
 cc=gcc
diff --git a/build.xml b/build.xml
index 692a115..41c3d4e 100644
--- a/build.xml
+++ b/build.xml
@@ -77,6 +77,7 @@ questions.

         <exec executable="${cc}" failonerror="true">
             <arg line="${arch64}"/>
+            <arg line="${archarm64}"/>
             <arg value="-I"/>
             <arg value="${javahome}/include"/>
             <arg value="-I"/>

the binary does show support for both architectures and starts an app just fine:

> lipo -detailed_info .../JavaAppLauncher
Fat header in: .../JavaAppLauncher
fat_magic 0xcafebabe
nfat_arch 2
architecture x86_64
    cputype CPU_TYPE_X86_64
    cpusubtype CPU_SUBTYPE_X86_64_ALL
    capabilities 0x0
    offset 16384
    size 54048
    align 2^14 (16384)
architecture arm64
    cputype CPU_TYPE_ARM64
    cpusubtype CPU_SUBTYPE_ARM64_ALL
    capabilities 0x0
    offset 81920
    size 70920
    align 2^14 (16384)

[Vampire]

Any news on this one?

[rednoah]

@sreilly I'd be happy to hire The Infinite Kind staff to work on arm64 support for appbundler.

[sreilly]

@rednoah thanks for the offer, but I think it should already be working with the patch above. I know I've got a universal launcher running, and just need to push any changes up. Sorry about the delay in incorporating @sfuerte's work!

[karlvr]

@sreilly I know what it's like not having any time. I've made a PR that might make it a little easier to close off this issue.

[rschmunk]

I have a weird situation regarding M1 support for my Java app that I hope people can help figure out.

I just downloaded and built a fresh copy of appbundler. Using it in my Java build process, I find that it does build a fat Java launcher for my app. However, the app still fails to launch and displays the "please install Java" dialog.

I added some extra diagnostics to main.m, and it reports that

2022-03-21 22:54:42.916520-0400 MyApp[35634:968012] int launch(char *, int, char **) dlopen failed: dlopen(/Library/Java/JavaVirtualMachines/zulu-17.jdk/Contents/Home/lib/libjli.dylib, 0x0001): tried: '/Library/Java/JavaVirtualMachines/zulu-17.jdk/Contents/Home/lib/libjli.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/libjli.dylib' (no such file)

In other words, even though the Java launcher is fat, it seems to be using the x86_64 code, which doesn't like the arm64 libjli.dylib in my Azul Zulu Java 17 install.

Even more confusingly, one of my app users says that the app is launching for him on an M1 Mac with an arm64 Azul Zulu Java 17.

Any ideas?

I have determined that if I build a skinny appbundler which is arm64 only, my app will launch on this M1 Mac.

[ijabz]

Hi rschmunk, how do you build fat instead skinny. I am currently having to one build on Intel Mac and one on M1 Mac to provide two skinny dmgs, with the user having to make sure they pick the right one.

[rschmunk]

@false, I downloaded the latest version of the code and simply built it as is on an M1 Mac. Using the lipo tool as described above, it reports that the launcher is fat.

[ijabz]

What do you pass as parameter to lipo i cant get it to work

[sreilly]

You can check the universal status of the launcher using the file command. From within the top level of the .app bundle folder:

% file Contents/MacOS/*
Contents/MacOS/Moneydance: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64
- Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64
- Mach-O 64-bit executable arm64]
Contents/MacOS/Moneydance (for architecture x86_64):    Mach-O 64-bit executable x86_64
Contents/MacOS/Moneydance (for architecture arm64): Mach-O 64-bit executable arm64

The problem is that all of the .dylib libraries that form the jvm also need to be fat binaries. I don't think this is the case with any of the prebuilt [open]jdk distributions. There is a lipo command you can run that will combine x86_64 and arm64 dylibs (from here):

lipo lib1.dylib lib2.dylib -output combined.dylib -create

You'll need to run a script to do that, combining an x86_64 and arm64 jvm. Beyond that you'll also probably need to unpack any .so/.dylib files from jars that contain native code to do the same. Hopefully all of the jar-embedded libraries contain both architectures too. I don't know what other problems might stand in the way, but this would at least be some progress. I guess the end goal is to have pre-built mac openjdks that are already universal.

[ijabz]

OK rschmunk running file shows that main application is universal, but of course the Java runtime it is using is not as I only create a jre for the platform I am running on, so maybe this is what you are seeing

@sreilly my knowledge of this is very limited but I was just hoping that a solution would be to allow two instances of jlink in the appbundler task and one could be used to generate M1 and one to generate Intel since you dont need to be on the platform to generate the runtime (e.g on my Windows PC I use the jlink command directly to create both Windows Jre from Windows Jdk and Linux jre from Linux jdk). So the bundled app would just contain two Java runtimes in the PlugIns folder and then the main application created would be able to automatically select the correct one based on its cpu.

[rschmunk]

What do you pass as parameter to lipo i cant get it to work

cd into your app package's Contents/MacOS directory and run

lipo -detailed_info MyAppName

or as @sreilly suggests

file MyAppName

[jlholt]

Surely someone has worked around the problem by now. What are the methods of changing this project's code that would allow it to work better?

[sreilly]

I have solved this with a script that uses the lipo command to combine two JREs that I generate using jlink.

I have another script in which I run jlink on adoptium JDKs to generate directories for each platform's JRE bundle. The result of that script is a directory for each jvm, including jvm-mac and jvm-macarm.

The next step is running the following script to combine them into a jvm-universal directory:

#!/bin/bash

# Script to combine an intel and arm64 JDK into a single universal JDK
# originally sourced from https://incenp.org/notes/2023/universal-java-app-on-macos.html

source build_settings

[ -f jvm-mac/Contents/Home/lib/libjava.dylib ] || die "Missing intel JRE"
[ -f jvm-macarm/Contents/Home/lib/libjava.dylib ] || die "Missing arm64 JRE"

echo "Creating universal JRE..."
rm -rf jvm-macuniversal
mkdir jvm-macuniversal
find jvm-macarm -type f | while read arm_file ; do
    noarch_file=${arm_file#jvm-macarm/}
    mkdir -p jvm-macuniversal/${noarch_file%/*}
    if file $arm_file | grep "Mach-O.\+arm64" ; then
        # Create universal binary from both x86_64 and arm64
        lipo -create -output jvm-macuniversal/$noarch_file jvm-mac/$noarch_file $arm_file
        if file $arm_file | grep executable ; then
            chmod 755 jvm-macuniversal/$noarch_file
        fi
    else
        # Not a file with binary code, copy it as it is
        cp $arm_file jvm-macuniversal/$noarch_file
    fi
done

echo "Packaging the JRE..."
(cd jvm-macuniversal/Contents
 rm -rf MacOS _CodeSignature)

I then use appbundler to make an app bundle with the jvm-universal JRE. This has worked in my limited testing so far. I haven't yet released an app that uses this bundle, but I hope to do so soon.

[Vampire]

Actually, this issue is solved in this project. The problem is within evolvedbinary/appbundler-maven-build, which does not reuse the build defined here, but only reuses the code and does its own compiling. If you build with the build here, you get a proper multi-arch binary. If you build with the build over there, you get an x86_64 only binary and that is what is available on Maven Central.