Apple Silicon (arm64) support for the Proffie tools package

[forsaoss]

The "arm-none-eabi-gcc" toolset with version "9-2020-q2-update" per the current "package_proffieboard_index.json" does not offer a native build of the tools package for Apple ARM64 arch (M1, M2, etc). We are 2+ years since public release of Apple ARM64 arch and humbly ask for a native build of this toolset.

Arduino IDE recently released 2.0.3 with a native Apple ARM64. [_Granted, various tools within its bundled arduino-cli -- like serial-discovery and mdns-discovery -- are actually also only built for x8664 arch on macOS with no ARM64 (separate issue to be taken up with them). This can be mitigated by building those Golang projects from source on macOS readily and easily though.]. It would be great if the board toolsets would support this arch as well.

I got to the point of compilation of ProffieOS only to have it fail due to bad CPU type of the executables.

Compilation error: fork/exec /Users/myUser/Library/Arduino15/packages/proffieboard/tools/arm-none-eabi-gcc/9-2020-q2-update/bin/arm-none-eabi-g++: bad CPU type in executable

[profezzorn]

I thought M1 macs were able to run x86 binaries just fine, albeit a bit slower?

[forsaoss]

@profezzorn with Rosetta, yes, but Rosetta is meant for as an aid to help in transition from x86 to ARM64, to fill the gap while developers get their stuff together and flip a compiler flag or two to build native or universal binaries. ;-) Going forth, Rosetta should only be for x86 apps that are old and/or no longer supported/developed at most. Any actively maintained software should absolutely consider offering native or universal binaries.

Rosetta is not guaranteed to stay around forever. Logically it doesn't make sense for it to especially considering Apple's separation from Intel and the fact they no longer offer Intel-based hardware. It was even rumored a little over a year ago that Rosetta would be removed from macOS, though this didn't happen.

Also see this over at the Apple Dev forums, where EOL for Rosetta was asked and the notion that once Apple does decide to remove Rosetta for whatever reason, developers will likely have a short window of time to deal with it. Again it's already been two years and any short window Apple gives will be justified in their eyes no doubt.

Finally Apple's own KB about installing and using Rosetta, particularly the paragraph:

In most cases, you won't notice any difference in the performance of an app that needs Rosetta. But you should contact the app developer to inquire about a version that can natively use the full power and performance of Apple silicon.

And so here we are. ;-) I have an M1 Max Mac Studio as my daily driver and a M2 MacBook Air. Neither have nor will have Rosetta installed because in my mind it simply shouldn't be needed at this stage, and I have nearly 100% native coverage for all the apps I use regularly. I only recently got into Lightsaber collection and currently have only one Proffie-based hilt which is from Sabertrio. Those guys recently released version 3 of their "pre-configuration" for ProffieOS 6.7 and I've been wanting to flash that and tinker a bit. I can't justify installing Rosetta for a super-niche hobby -- worst case, I hunt down and/or borrow an Intel box.

I'm personally quite computer-savvy, having been in the field in IT and dev roles since the late 90's. I'd be willing to compile the toolchains locally if that's feasible, just need to know how! Sorry if the info is out there already and my searching skills failed. Appreciate your help, and I'm sure other M1-using lightsaber geeks would too!

[profezzorn]

Next time I upgrade the arduino-proffieboard compiler, I will certainly look to see if an mac-arm64 toolchain is available. Last time I did it, it wasn't available, which is why there isn't one available now. (I didn't compile the toolchain myself, I just downloaded it from the arm-gcc website.)

Since an easy workaround exists, I'm not going to spend a lot of time on this particular problem. If you don't want to install rosetta that is your choice, but for now that is the recommended way to make the arduino-proffieboard plugin work on M1/M2 macs.

If you compile your own toolchain, and it works properly, and it's the same version of GCC as the rest of arduino-proffieboard uses, I'd be happy to add that to the arduino-proffieboard plugin so that other people can use it. I can't really provide a lot of advice on how to do that though, since I didn't actually do it myself for any of the other platforms.

[forsaoss]

Got it. Thanks.

[forsaoss]

Just one more thing: from the version "9-2020-q2-update" I infer this is from possible September 2020, is that right? And looks like the version was 9.3.1 actually which was sometime back in mid 2020, which pre-dates the first Apple Silicon Macs released later in the year. So that definitely explains why the binaries aren't built for arm64.

I see you didn't build the toolchain yourself and that's well and good; perhaps your source for the toolchain has re-built it since then, and perhaps it's as simple as updating your plugin json to reference that new package. Might not be a lot of time to spend. :)

Anyway, poking around in the x86 binaries I can pull out key bits of info to help with compilation, such as the entire configure command line, like this from gcc itself: /tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/src/gcc/configure --target=arm-none-eabi --prefix=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native --libexecdir=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/lib --infodir=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/install-native/arm-none-eabi --build=x86_64-apple-darwin10 --host=x86_64-apple-darwin10 --with-gmp=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/build-native/host-libs/usr --with-mpfr=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/build-native/host-libs/usr --with-mpc=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/build-native/host-libs/usr --with-isl=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/build-native/host-libs/usr --with-libelf=/tmp/jenkins-GCC-9-pipeline-200_20200521_1590053285/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-lstdc++ -lm' --with-pkgversion='GNU Arm Embedded Toolchain 9-2020-q2-update' --with-multilib-list=rmprofile,aprofile

Looks like that was built by some automated build process on Jenkins, back in May 2020. Do we know where that Jenkins is? Otherwise I will absolutely try to build the same in my environment. Thanks again.

[profezzorn]

All the offical arm-gcc toolchains builds and sources can be found here: https://developer.arm.com/downloads/-/gnu-rm

I'm guessing they will also have some information about how to build your own, but I have not looked for that. I suspect that "jenkins" refers to a continuous integration system.

[forsaoss]

Yes Jenkins is a CI system. I guess I worded that request poorly; I meant if you knew where the Jenkins environment itself actually was, who maintained it, etc, and if they have possibly produced newer builds of the tools.

Seems like the URL you just provided may actually be the answer I'm looking for so thank you very much for that! Although even the current release there is over a year old and still without arm64 for macOS, they have source tarballs there including for the 9-2020-q2-update from June 2020 that your plugin is using... so I guess... wish me luck?? I'll try building that and I'll let you know how that works out!

[forsaoss]

Well this is an immediate futile effort. I did download the source for 9-2020-q2-update from https://developer.arm.com/downloads/-/gnu-rm

And per their How-to-build-toolchain.pdf doc in section 2.1 it specifically calls for an an x86-based Apple Mac family machine. I proceeded with the build attempt anyway and it failed because of some x86-specific assembler code.

This is unfortunate but it's clearly not an issue for or about ProffieOS itself. I'll take this up with the folks at developer.arm.com

It's worth noting that https://developer.arm.com/downloads/-/gnu-rm is a deprecated page. https://developer.arm.com/Tools%20and%20Software/GNU%20Toolchain is the official page now. The actual latest release is 12.2 as of September this year and still does not have arm64 for macOS which I find pretty astounding and I'd hope has some deeper technical reason than the likes of too lazy or unwilling to update assembler code. https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads

Sorry to clutter up your issue tracker here with what amounts to third-party stuff, but hopefully this is useful to others that might come across this.

Thanks for you time! You can absolutely close this.

[profezzorn]

The only assembler code in GCC is the parts that generate code, not run code, so I think it's just a matter of making it compile. There may be minor issues, but in a broader sense it should just work. I think the instructions reflect what has been tested, not what is possible.