AppleSilicon (mac aarch64) support

[ArtRoman]

Need to add mac aarch64 support.

I was able to build native multiarch library by "make osx" with changing

osx: export CFLAGS += -I$(JAVA_HOME)/include/darwin -arch x86_64
osx: export LDFLAGS := -arch x86_64 -dynamiclib -framework JavaVM -framework IOKit -framework CoreFoundation

to this:

osx: export CFLAGS += -I$(JAVA_HOME)/include/darwin -arch x86_64 -arch arm64
osx: export LDFLAGS := -arch x86_64 -arch arm64 -dynamiclib  -framework IOKit -framework CoreFoundation

(also note removing JavaVM framework because it will cause error if official Apple Java 1.8 x86_64 is not installed)

So "file" utils shows next info:

% file build/resources/main/native/osx/libNRJavaSerial.jnilib 
build/resources/main/native/osx/libNRJavaSerial.jnilib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64:Mach-O 64-bit dynamically linked shared library arm64]
build/resources/main/native/osx/libNRJavaSerial.jnilib (for architecture x86_64):   Mach-O 64-bit dynamically linked shared library x86_64
build/resources/main/native/osx/libNRJavaSerial.jnilib (for architecture arm64):    Mach-O 64-bit dynamically linked shared library arm64

But this is not usable on Apple Silicon by default because it's arm, and loader requires native library to be named "libNRJavaSerialv8" or "libNRJavaSerial_legacy" for loading.

Replacing if(OSUtil.isARM()) { to if(OSUtil.isARM() && !OSUtil.isOSX()) { in NativeResource.java:107 fixes the issue and the library work well natively on M1 mac.

[MrDOS]

Hey, thank you for chiming in. I hope you saw #195, because I think you've covered a lot of the same ground. I split the macOS builds into two different targets, one for x86_64, and one for AArch64. TBH, I didn't realize you could pass the -arch flag to clang twice and have it do The Right Thing. Are you aware of any difference between passing -arch twice vs. combining the outputs with lipo(1)?

loader requires native library to be named "libNRJavaSerialv8" or "libNRJavaSerial_legacy" for loading.

Ah, shoot, as I mentioned in #195, I haven't actually tried my build yet (which is why I haven't converted it to a PR), so I hadn't noticed that the check for ARM occurs before the check for macOS. Good catch. (As I mentioned in #218, I'd like to replace all of those condition trees in the native loading code with table-based lookups anyway.)

[ArtRoman]

I've missed that issue and started from ground, but after about half an hour of research I've got universal library that works on M1 mac and on Intel-based mac. Thank you for the good structured project!

Are you aware of any difference between passing -arch twice vs. combining the outputs with lipo(1)?

https://gcc.gnu.org/onlinedocs/gcc/Darwin-Options.html says: Apple’s GCC on Darwin does create “fat” files if multiple -arch options are used; it does so by running the compiler or linker multiple times and joining the results together with lipo.

So it is done by default as I understand. Passing two params is simple way to achieve universal binary and required because without -arch param only single native arch will be built: aarch64 on M1 mac and x86_64 on intel mac.

Passing two arch is works even for intel-based macs if "Xcode 12.5" or "Command Line Tools for Xcode 12.5" installed, because it has toolchains to make arm64 binaries on intel host and x86_64 binaries on arm host.

But I've found an issue: my build of library is requires Mac OS X 11. Loading library on Mac OS X 10.14 crashes app with next error:

Dyld Error Message:
  Symbol not found: ___darwin_check_fd_set_overflow
  Referenced from: /private/var/folders/*/libNRJavaSerial.jnilib (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib

So your approach is more correct: build arm version for target arm64-apple-macos11, build intel version for target x86_64-apple-macos10.5, then call lipo to get Universal Binary as says apple: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

I just added

    rm resources/native/osx/libNRJavaSerialintel.jnilib \
         resources/native/osx/libNRJavaSerialarm64.jnilib

to remove duplicated libraries after merging them to single libNRJavaSerial.jnilib.

The result library is working well on M1 mac and also on older intel macs.

[ArtRoman]

Also need to add

targetCompatibility = JavaVersion.VERSION_1_8
sourceCompatibility = JavaVersion.VERSION_1_8

into build.gradle for ability to use library on Java 1.8. Otherwise there is an error on OpenJDK 1.8: java.lang.UnsupportedClassVersionError: gnu/io/CommPortIdentifier has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 after building library with JDK11.

[twdorris]

@ArtRoman This thread is one of those classic examples of what makes the internet hive-mind work! Thank you so much for this write up! Saved me countless hours and some of the few precious hairs I have left on my head.

@MrDOS Are there any plans to bundle this up into a PR? If not, I may take a stab at it if you want. Everything above worked 100% for me in migrating projects from my old 2016 Intel Mac running Catalina to a new 2021 M1 running Monterey.

[trevorbattle]

Bumping, I also followed the steps above and was able to get everything working on a 2021 M1 running Ventura. I had to use Java 11 to use the gradlew script to make the JAR, but it all compiled and worked.

@twdorris I could also bundle it up, if it is considered production ready now. But... I do not have the ability to test it all for Intel Macs. I haven't tried using this JAR on any other architectures. I use nvjavaserial on RPi for ZigBee, but I haven't messed with the older JAR there that is working.

[tdorris]

@trevorbattle I can test on both Intel and Arm Macs if that helps. And, of course, Intel Windows and Linux. If pressed, I could probably pull off Arm Window and Linux but I'm not exactly setup for that yet.

[trevorbattle]

Ok. So I have the working build for Arm Mac, although I didn't do the cross compiling so my "build" isn't complete. This defeats the purpose a bit, as you can't deploy the same JAR on multiple architectures.

I could conceivably build a working jar by combining the "native" folders of different builds, but not sure that would be something useful to more than a few people.

Happy to discuss and share my working one though, if anybody is desperate.

[twdorris]

@trevorbattle Yeah, I did the same here. I re-built as many versions (all the ARMs and Linux 32/64) as I could but didn't have any good way to build the various FreeBSD flavors without installing more VMs. I also hacked in some code to allow custom baud rates on Mac/Linux because I need that functionality but like you said, I'm not sure how many others really do so probably not worth a commit unless someone else says "Hey, I really need 15625 baud support!" LOL.