Add Apple M1 build.

[ppalaga]

Blocked by https://github.com/actions/virtual-environments/issues/2187

[syhily]

Since the mvnd is written in Java, we could provide a jvm based distribute as a universal version.

[ppalaga]

we could provide a jvm based distribute as a universal version.

That exists already. Download whichever build and use mvnd.sh instead of mvnd. The shell script starts a full JVM on the client side so it is a bit slower than the native client of course.

[syhily]

I have changed the homebrew install scripts for using the mvnd.sh instead of mvnd on Apple M1. Could this PR be merged into the master.

[gnodet]

The PR for the home-brew recipe has been merged.

[ppalaga]

BTW, this GraalVM issue states that native executables for mac x86_64 are runnable on Apple silicon using Rosetta Translation Environment

It would be interesting to compare the speed of the translated binary with mvnd.sh + JVM. BTW, isn't the JVM on Mac AArch64 also just a Rosetta translation?

[ppalaga]

Reopening, because we want a Mac@AArch64 build when all the parts are available.

This is also blocked by https://github.com/oracle/graal/issues/2666

[davsclaus]

There is also jansi

~/workspace/camel master ❯ mvnd clean install -P fastinstall
Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.mvndaemon.mvnd.client.DefaultClient.main(DefaultClient.java:91)
Caused by: java.lang.RuntimeException: Unable to load jansi native library
    at org.fusesource.jansi.internal.JansiLoader.initialize(JansiLoader.java:62)
    at org.fusesource.jansi.internal.CLibrary.<clinit>(CLibrary.java:30)
    ... 1 more
Caused by: java.lang.Exception: No native library found for os.name=Mac, os.arch=aarch64, paths=[/opt/homebrew/Cellar/mvnd/0.4.2/libexec/mvn/lib/jansi-native/Mac/aarch64:/opt/homebrew/Cellar/mvnd/0.4.2/libexec/mvn/lib/jansi-native:/Users/davsclaus/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
    at org.fusesource.jansi.internal.JansiLoader.loadJansiNativeLibrary(JansiLoader.java:333)
    at org.fusesource.jansi.internal.JansiLoader.initialize(JansiLoader.java:60)
    ... 2 more

[gnodet]

There is also jansi

~/workspace/camel master ❯ mvnd clean install -P fastinstall
Exception in thread "main" java.lang.ExceptionInInitializerError
  at org.mvndaemon.mvnd.client.DefaultClient.main(DefaultClient.java:91)
Caused by: java.lang.RuntimeException: Unable to load jansi native library
  at org.fusesource.jansi.internal.JansiLoader.initialize(JansiLoader.java:62)
  at org.fusesource.jansi.internal.CLibrary.<clinit>(CLibrary.java:30)
  ... 1 more
Caused by: java.lang.Exception: No native library found for os.name=Mac, os.arch=aarch64, paths=[/opt/homebrew/Cellar/mvnd/0.4.2/libexec/mvn/lib/jansi-native/Mac/aarch64:/opt/homebrew/Cellar/mvnd/0.4.2/libexec/mvn/lib/jansi-native:/Users/davsclaus/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
  at org.fusesource.jansi.internal.JansiLoader.loadJansiNativeLibrary(JansiLoader.java:333)
  at org.fusesource.jansi.internal.JansiLoader.initialize(JansiLoader.java:60)
  ... 2 more

This would require a fix for https://github.com/multiarch/crossbuild/issues/51 so that jansi could have a M1 version.

[electrum]

I was able to run mvnd on M1 by using an x86 JDK:

JAVA_HOME=/tmp/zulu16.32.15-ca-jdk16.0.2-macosx_x64 mvnd clean install -DskipTests

However, even after the daemon was warmed up, it was still slower than running mvn using the native ARM JDK. My test run was only building four sequential modules, so there might still be benefit to non-native mvnd on a larger project.

[electrum]

I think we can close this as it was fixed in #502. I’ve been happily using the 0.7.0 release successfully on my M1 Mac mini!

[davsclaus]

I am on M1 and mvnd 0.7.0 works there.

[gnodet]

Well, it works, but it's still using the Apples' Rosetta 2 translation layer. I'd rather keep that one opened until GraalVM supports M1 directly.

[ppalaga]

There is some update in https://github.com/oracle/graal/issues/2666#issuecomment-1010183689 It says

JDK17 / labs-openjdk-17 supports darwin-aarch64 today

but it is not clear to me whether the changes are released already and which release it is.

[ppalaga]

This comment https://github.com/oracle/graal/issues/2666#issuecomment-1074884020 states

I'm pleased to announce that we uploaded the first snapshot containing darwin-aarch64: https://github.com/graalvm/graalvm-ce-dev-builds/releases/tag/22.1.0-dev-20220321_2332

The following components are available for this platform:

• JVM with libgraal
• native-image
• JavaScript
• Espresso

This would make it at least possible to compile locally on a darwin-aarch64 machine.

[orpiske]

This comment oracle/graal#2666 (comment) states

I'm pleased to announce that we uploaded the first snapshot containing darwin-aarch64: https://github.com/graalvm/graalvm-ce-dev-builds/releases/tag/22.1.0-dev-20220321_2332 The following components are available for this platform:

• JVM with libgraal
• native-image
• JavaScript
• Espresso

This would make it at least possible to compile locally on a darwin-aarch64 machine.

I am trying it to see how it goes 🤞

[orpiske]

@ppalaga it built fine. I tried building Camel with it (cached) and it went fine. Performance was nice too (10 core M1 Pro building it roughly with the same duration as a Ryzen 9 3900XT).

[orpiske]

I know support is not official yet, but sharing my experience in the hope it will eventually be useful for when there is an official release.

One thing I noticed is that the build for ARM doesn't seem to respect the mvnd.threads setting or the -T flag. Using -T 0.5C here, still causes it to use all cores.

[ppalaga]

A new issue for OSX arm64 runners hosted by GitHub was recently added to GH public roadmap: https://github.com/github/roadmap/issues/528

I think this issue won't move without that one.

[ppalaga]

Citing from https://github.com/actions/virtual-environments/issues/2187#issuecomment-1164711613

Have contacted GitHub support and they hope to have it in place by end of FY2022.

[orpiske]

Citing from actions/virtual-environments#2187 (comment)

Have contacted GitHub support and they hope to have it in place by end of FY2022.

Ouch, that's quite a long time. I'm wondering if it would be feasible to provide mvnd on homebrew. I don't know much about it, so I hope I am not talking non-sense here, but if homebrew builds from source that would solve the need for a while.

[mthmulders]

[...] I'm wondering if it would be feasible to provide mvnd on homebrew. I don't know much about it, so I hope I am not talking non-sense here, but if homebrew builds from source that would solve the need for a while.

AFAIK, Homebrew doesn't provide a build service. They have manifests (in fact, Ruby files) that describe how to download (and optionally build) software on the users machine.

[orpiske]

[...] I'm wondering if it would be feasible to provide mvnd on homebrew. I don't know much about it, so I hope I am not talking non-sense here, but if homebrew builds from source that would solve the need for a while.

AFAIK, Homebrew doesn't provide a build service. They have manifests (in fact, Ruby files) that describe how to download (and optionally build) software on the users machine.

Yeah, that's what I was thinking of (sorry, I did not explain it well). Just have it build on the user's computer with the help of brew.

[kameshsampath]

I am still waiting for this on M1, doing a build from source on M1 will help or I need to wait for official release?

[ppalaga]

doing a build from source on M1 will help or I need to wait for official release?

@orpiske 's script is still be the best way to get a working native mvnd on Apple Silicon Mac https://www.orpiske.net/2022/07/try-apache-maven-daemon/

[mprimeaux]

brew install mvndaemon/homebrew-mvnd/mvnd works on both my Mac Studio (M1 Ultra) and MacBook Pro (M1 Max) for Java 9 and greater.

Unfortunately, Java 8 (Temurin Open JDK) fails with the following:

JAVA_HOME is not set. Setting JAVA_HOME to the location of your Java installation.
JAVA_HOME is set to /Library/Java/JavaVirtualMachines/temurin-8.jdk/Contents/Home.
Using 'mvnd' to build workload.
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/mvndaemon/mvnd/client/DefaultClient has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)
make: *** [compile] Error 1

[ppalaga]

@mprimeaux for running mvnd.sh with Java 8, please see https://github.com/apache/maven-mvnd/issues/669#issuecomment-1205335100 and https://github.com/apache/maven-mvnd/issues/547#issuecomment-1004889625

[gnodet]

@mprimeaux Also, a candidate release of mvnd 0.8.2 with M1 builds is available for review at https://dist.apache.org/repos/dist/dev/maven/mvnd/0.8.2/. It would be nice to have some feedback for the M1 builds.

[ppalaga]

How were those M1 builds produced, @gnodet ?

[gnodet]

How were those M1 builds produced, @gnodet ?

On a Mac mini M1 that MacStadium has freely provided to me for the maven project.

[mprimeaux]

@gnodet mvnd 0.8.2 works on all of my M1 machines (M1 Ultra and M1 Max) using Java 8 and greater (Temurin JDK). A sincere thanks for your comments above.

[orpiske]

@mprimeaux Also, a candidate release of mvnd 0.8.2 with M1 builds is available for review at https://dist.apache.org/repos/dist/dev/maven/mvnd/0.8.2/. It would be nice to have some feedback for the M1 builds.

I've just tested the 0.8.2 build with Camel and it works nicely!

[mprimeaux]

It seems the brew formula for mvnd was updated to 0.8.2. I'm on an Apple M1 Ultra Silicon.

Oddly, however, I'm still receiving the same exception as before after installing the updated version.

❯ brew list mvnd                                                                                                                                           ─╯
/opt/homebrew/Cellar/mvnd/0.8.2/bin/mvnd
/opt/homebrew/Cellar/mvnd/0.8.2/bin/mvnd-bash-completion.bash
/opt/homebrew/Cellar/mvnd/0.8.2/libexec/bin/ (2 files)
/opt/homebrew/Cellar/mvnd/0.8.2/libexec/conf/ (3 files)
/opt/homebrew/Cellar/mvnd/0.8.2/libexec/mvn/ (85 files)

Against the Temurin 8 JDK:

❯ mvnd --version                                                                                                                                           ─╯
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/mvndaemon/mvnd/client/DefaultClient has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)

I decided to downloaded version 0.8.2 again and test as per @gnodet here and this version works great against the same Java 8 source that failed above.

I'm not yet sure what's going on but thought I'd post it here before I investigate.

[gnodet]

When I released 0.8.2, i used the usual formula which had not been updated wrt to the new M1 binaries. I think you still have the mvnd being replaced by the mvnd.sh script which is not a native library (that was the case for arm platforms). Can you uninstall mvnd using homebrew and reinstall or force an update somehow ? I just updated the formulae 1 hour ago: https://github.com/mvndaemon/homebrew-mvnd/commit/6e3a40c6928c9f1c800942ac673bd2c7a5e65616

[mprimeaux]

Yes. Let me try that now.

UPDATED: Uninstalling and reinstalling worked. Thanks again!

[Hadi83mars402402]

مشكورر

[gzm55]

@gnodet i tested the universe binary combining two archs of amd64 and arm64 on macos, the result binary works well on intel mac, and should work as well on arm mac. the zip file size is increased from 20M to 29M. how about releasing the universe native binary for macos?

[gnodet]

@gnodet i tested the universe binary combining two archs of amd64 and arm64 on macos, the result binary works well on intel mac, and should work as well on arm mac. the zip file size is increased from 20M to 29M. how about releasing the universe native binary for macos?

That would remove a specific platform and ease releases as the M1 builds have to be done and uploaded manually. How do we tell graalvm to build a universal binary ?

[gzm55]

@gnodet i tested the universe binary combining two archs of amd64 and arm64 on macos, the result binary works well on intel mac, and should work as well on arm mac. the zip file size is increased from 20M to 29M. how about releasing the universe native binary for macos?

That would remove a specific platform and ease releases as the M1 builds have to be done and uploaded manually. How do we tell graalvm to build a universal binary ?

@gnodet graalvm cannot cross compile the native binary right now. what i have done is manually download two macos distributions, extract, then run

lipo -create -output /path/to/universe/binary/mvnd maven-mvnd-1.0-m6-m39-darwin-{amd64,aarch64}/bin/mvnd

and repack zip at last.

This process still requires a special environment to build the M1 binary and combine a universe macos distribution.

[gnodet]

@gnodet i tested the universe binary combining two archs of amd64 and arm64 on macos, the result binary works well on intel mac, and should work as well on arm mac. the zip file size is increased from 20M to 29M. how about releasing the universe native binary for macos?

That would remove a specific platform and ease releases as the M1 builds have to be done and uploaded manually. How do we tell graalvm to build a universal binary ?

@gnodet graalvm cannot cross compile the native binary right now. what i have done is manually download two macos distributions, extract, then run

lipo -create -output /path/to/universe/binary/mvnd maven-mvnd-1.0-m6-m39-darwin-{amd64,aarch64}/bin/mvnd

and repack zip at last.

This process still requires a special environment to build the M1 binary and combine a universe macos distribution.

This makes the release process even more complicated. What's the real benefit ?

[gzm55]

@gnodet i tested the universe binary combining two archs of amd64 and arm64 on macos, the result binary works well on intel mac, and should work as well on arm mac. the zip file size is increased from 20M to 29M. how about releasing the universe native binary for macos?

That would remove a specific platform and ease releases as the M1 builds have to be done and uploaded manually. How do we tell graalvm to build a universal binary ?

@gnodet graalvm cannot cross compile the native binary right now. what i have done is manually download two macos distributions, extract, then run

lipo -create -output /path/to/universe/binary/mvnd maven-mvnd-1.0-m6-m39-darwin-{amd64,aarch64}/bin/mvnd

and repack zip at last. This process still requires a special environment to build the M1 binary and combine a universe macos distribution.

This makes the release process even more complicated. What's the real benefit ?

yes, the release process need to do some little refactor. with this feature, the macos developer can more easily select the correct distribution file.

[ppalaga]

As per https://github.com/actions/runner-images/issues/8439

Apple Silicon (M1) powered macOS [GH actions runners] are now available in public beta!

So we could extend our testing and release matrix to include macos-latest-xlarge or macos-13-xlarge

[ppalaga]

When reading https://github.com/actions/runner-images/issues/8439 properly, it looks like the named images are not yet a part of the GHA free tier.

[radeva]

@ppalaga You can try FlyCI's M1 and M2 runners. They are on average 2x faster and 2x cheaper than GitHub's AND we have a free tier for OSS projects (see below).

Install Instructions

1. Install FlyCI app and
2. Easily replace one line of code and start using FlyCI runners:

jobs:
 ci:
-    runs-on: macos-latest
+    runs-on: flyci-macos-large-latest-m1
   steps:
   - name: 👀 Checkout repo
     uses: actions/checkout@v4

500 mins/month Free for Public Repos

Since your repo is public, FlyCI offers 500 mins/month of free M1 runner usage with the flyci-macos-large-latest-m1 runner for public projects.

Don't hesitate to contact us in case the free tier doesn't suit your needs or you experience any issues with the runners. Our team is here to support you!

Best Regards, Veselina Radeva Product Manager at FlyCI

[gnodet]

MacOS binaries are now provided since https://github.com/apache/maven-mvnd/pull/1027