Building CLIJ2-FFT for Apple Silicon M1 (arm64) on MacOS 11.4 Big Sur

[psobolewskiPhD]

I tried running Ext.CLIJx_deconvolveRichardsonLucyFFT() on my MBP with Apple Silicon M1 (MacOS 11.4) Predictably, it failed to architecture mismatch: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: /Applications/Fiji.app/lib/macosx/libjniclij2fftWrapper.dylib: dlopen(/Applications/Fiji.app/lib/macosx/libjniclij2fftWrapper.dylib, 1): no suitable image found. Did find: /Applications/Fiji.app/lib/macosx/libjniclij2fftWrapper.dylib: mach-o, but wrong architecture /Applications/Fiji.app/lib/macosx/libjniclij2fftWrapper.dylib: mach-o, but wrong architecture

So I thought instead of doing real work, I would try to contribute here and build it, as I had once done for JOCL. I made a fork (https://github.com/psobolewskiPhD/clij2-fft) and took a look at the scripts. I really don't understand the structure with the two different cppbuild.sh... Anyhow, I installed clFFT native arm64 for Big Sur from homebrew. I edited clij-fft/cppbuild.sh to account for arm64—to be consistent with cmake env variables. Then I edited the second cppbuild.sh to replace the hard path to the path where clFFT is installed by homebrew (/opt/homebrew/opt/clfft/lib). (I ran into an not obvious snag with a comment line, but got it resolved). Then in 'clij2-fft/native' I ran {bash cppbuild.sh} which seemed to succeed: -- Generating done -- Build files have been written to: /Users/piotrsobolewski/Dev/clij2-fft/native/clij2fft/cppbuild ~/Dev/clij2-fft/native Based on Roberts_build_notes.md:

cd clij2fft
cd ccpbuild
make

It maybe worked? 5 warnings generated. [100%] Linking CXX shared library libclij2fft.dylib ld: warning: directory not found for option '-L/Users/haase/code/ops-experiments/ops-experiments-opencl/native' [100%] Built target clij2fft 5 Jun 18:09 libclij2fft.dylib* Here's a link to the dylib: https://www.dropbox.com/s/64chkqi7pwc5c1h/libclij2fft.dylib.zip?dl=0 But I'm not sure what I need to do next? I don't have a libjniclij2fftWrapper.dylib

Edit: I think it did work? The dylib does seem to be arm64 anyways.

> lipo -info libclij2ff.dylib
Non-fat file: libclij2fft.dylib is architecture: arm64

[bnorthan]

"I really don't understand the structure with the two different cppbuild.sh"

The top level cppbuild.sh is based on a this one from the javacpp presets project. It sets up variables for cmake, the OS, etc then calls the low level cppbuild.sh for each project.

In javacpp presets there are several projects. In clij2fft there is only one. However both follow the same pattern, high level cppbuild.sh sets things up, low level ones build each project.

I am wondering if you need to change the top level cppbuild.sh to differentiate between Apple Silicon M1 and older Apple systems? Perhaps not because the location of the files will still be the same?

What we will have to do (I think) is differentiate between the different types of Mac when we prepare the libs for distribution.

Currently we are building and arching here. We have linux64, macosx, and win64, do we have to add a fourth for Apple M1?

This gets tricky because these libs are placed in the Fiji.app directory, and Fiji looks for them in these specific locations. If Fiji aware of Apple M1?

[psobolewskiPhD]

I am wondering if you need to change the top level cppbuild.sh to differentiate between Apple Silicon M1 and older Apple systems? Perhaps not because the location of the files will still be the same?

Yup, i did this: https://github.com/psobolewskiPhD/clij2-fft/blob/06a759b720afc59b40aa4efc8951f8f1b04efdd7/native/cppbuild.sh#L33

And in the second build script I point to the homebrew clFFT: https://github.com/psobolewskiPhD/clij2-fft/blob/06a759b720afc59b40aa4efc8951f8f1b04efdd7/native/clij2fft/cppbuild.sh#L33

This part all builds and runs—I can use the lib in python with no issue. It's the java stuff that I couldn't figure out.

[bnorthan]

"It's the java stuff that I couldn't figure out."

I'm not sure how familiar you are with maven, but to build the java part you just go to the top level directory of the project and type 'mvn'.

In the pom.xml file we specify that the javacpp plugin should be used to build the java wrapper. See here

Here is the wrapper specification.

Things are a bit inconsistent because there are multiple places you can specify link paths and we've set things up a bit differently for each. In fact for the current Mac build there is no Mac Specific Platform annotation in the wrapper.

I think the build still works because the library path is defined in the pom. I don't know for sure though because I haven't built the Mac version myself.

[psobolewskiPhD]

I have no idea what maven/mvn is 😢 I basically just have the normal c-based toolchains (cmake, gcc, clang, etc) plus now conda on the python side. Honestly, i've kinda taken to python and I have the deconvolution running on that side (toy data, but still), so I'm not sure how much time I can find for the java side of this now to get deeper down the rabbit hole.

[bnorthan]

Yeah. If you do not use Maven and are now using Python I would not bother with the java stuff at all.

These types of loosely organization collaborations work best when each person is focusing on the part they are actually using.

[psobolewskiPhD]

I doubt i will totally abandon Fiji, so CLIJ is still important to me, but probably this is a bit more back burner. I'm more then willing to test anything that needs M1 and I will try and take a look at maven when I get a chance.