saalfeldlab / n5-utils

simple standalone BigDataViewer for multiple N5 (or HDF5) datasets
Other
8 stars 16 forks source link

blosc support broken after install on macos12 #25

Open bogovicj opened 2 years ago

bogovicj commented 2 years ago

@avweigel observed this issue was fixed after adding an appropriate jna.library.path to the run scripts

-Djna.library.path="<the-path>" \

We may need to add this to the install script for new macos versions.

see also https://github.com/glencoesoftware/bioformats2raw/issues/101#issuecomment-879797175

mkitti commented 2 years ago

@avweigel said today that this may still be an issue.

Based on this output,

$ brew install c-blosc
$ brew list -v c-blosc
/opt/homebrew/Cellar/c-blosc/1.21.1/INSTALL_RECEIPT.json
/opt/homebrew/Cellar/c-blosc/1.21.1/.brew/c-blosc.rb
/opt/homebrew/Cellar/c-blosc/1.21.1/include/blosc-export.h
/opt/homebrew/Cellar/c-blosc/1.21.1/include/blosc.h
/opt/homebrew/Cellar/c-blosc/1.21.1/README.md
/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.1.dylib
/opt/homebrew/Cellar/c-blosc/1.21.1/lib/pkgconfig/blosc.pc
/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.dylib
/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.1.21.1.dylib
/opt/homebrew/Cellar/c-blosc/1.21.1/lib/libblosc.a

We should add java -Djna.library.path="/opt/homebrew/Cellar/c-blosc/1.21.1/lib" to the scripts.

mkitti commented 2 years ago

Another issue here is that JNA needs to be updated to version 5 since JNA needs to load a JNI library.

See https://github.com/scijava/pom-scijava/issues/201 https://github.com/scijava/pom-scijava/issues/163

mkitti commented 2 years ago

Adding <jna.version>5.11.0</jna.version> to <properties> helped.

% git diff
diff --git a/pom.xml b/pom.xml
index 00f38e4..f2dc8c6 100644
--- a/pom.xml
+++ b/pom.xml
@@ -115,6 +115,7 @@
                <jacoco-maven-plugin.version>0.8.6</jacoco-maven-plugin.version>

                <cisd.jhdf5.version>19.04.0</cisd.jhdf5.version>
+               <jna.version>5.11.0</jna.version>
                <n5.version>2.4.0</n5.version>
                <n5-imglib2.version>3.5.1</n5-imglib2.version>
                <n5-hdf5.version>1.4.2-SNAPSHOT</n5-hdf5.version>
mkitti commented 2 years ago
Exception excerpt without changing `jna.version` to `5.11.0`. Expand for full stack trace: ``` % ./n5-view -i ~/Documents/t1.n5 -d data Compression 'org.janelia.saalfeldlab.n5.blosc.BloscCompression' could not be registered because: java.lang.UnsatisfiedLinkError: /private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp: dlopen(/private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp, 0x0001): tried: '/private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')) at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) at java.lang.Runtime.load0(Runtime.java:811) at java.lang.System.load(System.java:1088) at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) at com.sun.jna.Native.(Native.java:190) at org.blosc.IBloscDll.(IBloscDll.java:15) at org.blosc.JBlosc.init(JBlosc.java:30) at org.blosc.JBlosc.(JBlosc.java:24) at org.janelia.saalfeldlab.n5.blosc.BloscCompression.(BloscCompression.java:70) ``` ``` % ./n5-view -i ~/Documents/t1.n5 -d data Compression 'org.janelia.saalfeldlab.n5.blosc.BloscCompression' could not be registered because: java.lang.UnsatisfiedLinkError: /private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp: dlopen(/private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp, 0x0001): tried: '/private/var/folders/5j/phv_vfc97m967ww98fbvtmfr0000gq/T/jna-816709643/jna2387896126297747561.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')) at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) at java.lang.Runtime.load0(Runtime.java:811) at java.lang.System.load(System.java:1088) at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) at com.sun.jna.Native.(Native.java:190) at org.blosc.IBloscDll.(IBloscDll.java:15) at org.blosc.JBlosc.init(JBlosc.java:30) at org.blosc.JBlosc.(JBlosc.java:24) at org.janelia.saalfeldlab.n5.blosc.BloscCompression.(BloscCompression.java:70) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.janelia.saalfeldlab.n5.CompressionAdapter.update(CompressionAdapter.java:82) at org.janelia.saalfeldlab.n5.CompressionAdapter.getJsonAdapter(CompressionAdapter.java:166) at org.janelia.saalfeldlab.n5.AbstractGsonReader.(AbstractGsonReader.java:58) at org.janelia.saalfeldlab.n5.N5FSReader.(N5FSReader.java:116) at org.janelia.saalfeldlab.n5.N5FSReader.(N5FSReader.java:136) at org.janelia.saalfeldlab.N5Factory.createN5FS(N5Factory.java:259) at org.janelia.saalfeldlab.N5Factory.createN5(N5Factory.java:207) at org.janelia.saalfeldlab.N5Factory.createN5Reader(N5Factory.java:186) at org.janelia.saalfeldlab.View.call(View.java:281) at org.janelia.saalfeldlab.View.call(View.java:175) at picocli.CommandLine.executeUserObject(CommandLine.java:1853) at picocli.CommandLine.access$1100(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2255) at picocli.CommandLine$RunLast.handle(CommandLine.java:2249) at picocli.CommandLine$RunLast.handle(CommandLine.java:2213) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2080) at picocli.CommandLine.execute(CommandLine.java:1978) at org.janelia.saalfeldlab.View.main(View.java:514) java.lang.NullPointerException at org.janelia.saalfeldlab.n5.CompressionAdapter.deserialize(CompressionAdapter.java:145) at org.janelia.saalfeldlab.n5.CompressionAdapter.deserialize(CompressionAdapter.java:56) at com.google.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:69) at com.google.gson.Gson.fromJson(Gson.java:932) at com.google.gson.Gson.fromJson(Gson.java:1003) at com.google.gson.Gson.fromJson(Gson.java:975) at org.janelia.saalfeldlab.n5.GsonAttributesParser.parseAttribute(GsonAttributesParser.java:78) at org.janelia.saalfeldlab.n5.AbstractGsonReader.getDatasetAttributes(AbstractGsonReader.java:96) at org.janelia.saalfeldlab.n5.N5Reader.datasetExists(N5Reader.java:304) at org.janelia.saalfeldlab.View.datasetN(View.java:597) at org.janelia.saalfeldlab.View.call(View.java:290) at org.janelia.saalfeldlab.View.call(View.java:175) at picocli.CommandLine.executeUserObject(CommandLine.java:1853) at picocli.CommandLine.access$1100(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2255) at picocli.CommandLine$RunLast.handle(CommandLine.java:2249) at picocli.CommandLine$RunLast.handle(CommandLine.java:2213) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2080) at picocli.CommandLine.execute(CommandLine.java:1978) at org.janelia.saalfeldlab.View.main(View.java:514) ```
mkitti commented 2 years ago

Once we update jna.version to 5.11.0 but without setting jna.library.path we get the following error. Note the difference in the library it is trying to load. The error stack above is complaining about the inability to the find the JNI bindings for JNA. The error stack below is complaining about not being able to locate libblosc.

% ./n5-view -i ~/Documents/t1.n5 -d data
Compression 'org.janelia.saalfeldlab.n5.blosc.BloscCompression' could not be registered because:
java.lang.UnsatisfiedLinkError: Unable to load library 'blosc':
dlopen(libblosc.dylib, 0x0009): tried: '/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/bin/./libblosc.dylib' (no such file), 'libblosc.dylib' (no such file), '/usr/lib/libblosc.dylib' (no such file), '/Users/kittisopikulm/Documents/src/n5-utils/libblosc.dylib' (no such file)
dlopen(libblosc.dylib, 0x0009): tried: '/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/bin/./libblosc.dylib' (no such file), 'libblosc.dylib' (no such file), '/usr/lib/libblosc.dylib' (no such file), '/Users/kittisopikulm/Documents/src/n5-utils/libblosc.dylib' (no such file)
dlopen(/Users/kittisopikulm/Library/Frameworks/blosc.framework/blosc, 0x0009): tried: '/Users/kittisopikulm/Library/Frameworks/blosc.framework/blosc' (no such file), '/System/Library/Frameworks/blosc.framework/blosc' (no such file)
dlopen(/Library/Frameworks/blosc.framework/blosc, 0x0009): tried: '/Library/Frameworks/blosc.framework/blosc' (no such file), '/System/Library/Frameworks/blosc.framework/blosc' (no such file)
dlopen(/System/Library/Frameworks/blosc.framework/blosc, 0x0009): tried: '/System/Library/Frameworks/blosc.framework/blosc' (no such file)
mkitti commented 2 years ago

Upon doing so, the image opens properly.

% ./n5-view -i ~/Documents/t1.n5 -d data
N5FSReader[basePath=/Users/kittisopikulm/Documents/t1.n5] : data, [1.0, 1.0, 1.0], [0.0, 255.0], [0.0, 0.0, 0.0], Num axes: 3
axes permutation: [0, 1, 2] -> [0, 1, 2]
data 3d-affine: (1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0)
./n5-view: line 11: -XX:+UseConcMarkSweepGC: command not found

There is an extra -XX:+UseConcMarkSweepGC line at the end in this case. That is created when I try to install over an existing install.