search

luben / zstd-jni

JNI binding for Zstd
Other
840 stars 166 forks source link

MacOS 10.14.6 incompatible dylib binary #321

Open chris9182 opened 1 month ago

chris9182 commented 1 month ago

We have customers that don't want to upgrade their Intel Mac machines from MacOS 10.14.6 due to the dropped support for 32-bit applications. The bundled binary /darwin/x86_64/libzstd-jni-1.5.6-3.dylib seems to be incompatible with this version though.

We get the following cause for a java.lang.NoClassDefFoundError exception:

Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-31295224134339030627.dylib: dlopen(/private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-31295224134339030627.dylib, 1): no suitable image found.  Did find:
    /private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-31295224134339030627.dylib: cannot load 'libzstd-jni-1.5.6-31295224134339030627.dylib' (load command 0x80000034 is unknown)
    /private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-31295224134339030627.dylib: cannot load 'libzstd-jni-1.5.6-31295224134339030627.dylib' (load command 0x80000034 is unknown)
no zstd-jni-1.5.6-3 in java.library.path: [some_user_path]:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
Unsupported OS/arch, cannot find /darwin/x86_64/libzstd-jni-1.5.6-3.dylib or load zstd-jni-1.5.6-3 from system libraries. Please try building from source the jar or providing libzstd-jni-1.5.6-3 in your system. [in thread "Thread-12"]
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
    at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
    at java.base/java.lang.System.loadLibrary(System.java:1993)
    at com.github.luben.zstd.util.Native$1.run(Native.java:70)
    at com.github.luben.zstd.util.Native$1.run(Native.java:68)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
    at com.github.luben.zstd.util.Native.loadLibrary(Native.java:68)
    at com.github.luben.zstd.util.Native.load(Native.java:155)
    at com.github.luben.zstd.util.Native.load(Native.java:86)
    at com.github.luben.zstd.ZstdCompressCtx.<clinit>(ZstdCompressCtx.java:12)
...

Online research indicates that:

load command 0x80000034 is unknown

signifies that the binary was built to target a MacOS version of >= 10.15.

Is there the possibility for you to include a binary that is compiled to be compatible with previous MacOS versions? We assume that some of our customers may stay on this version for a while due to the 32-bit compatibility. The version also does not seem too old, as 10.15 was released on June 3, 2019.

luben commented 1 month ago

Thanks for the analysis. Let me see if I can target older version. Are you able to test it if I provide you with a .dylib file?

chris9182 commented 1 month ago

I can, but this will take a few days after I have the file.

You're welcome, thank you for responding so quickly and having a look.

luben commented 1 month ago

OK, turns out we were already using -target x86_64-apple-macos10.12. I have added also -mmacosx-version-min=10.12 and released v1.5.6-4 with that build change. Please test and tell if it helps

chris9182 commented 1 month ago

We did test v1.5.6-4 again on that system and are unfortunately still getting the same error:

Exception in thread "Thread-7" java.lang.UnsatisfiedLinkError:
/private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-411923576037694246901.dylib: dlopen(/private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-411923576037694246901.dylib, 1): no suitable image found.  Did find:
    /private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-411923576037694246901.dylib: cannot load 'libzstd-jni-1.5.6-411923576037694246901.dylib' (load command 0x80000034 is unknown)
    /private/var/folders/54/sbcw2m010vx32hrjp85jcdh00000gn/T/libzstd-jni-1.5.6-411923576037694246901.dylib: cannot load 'libzstd-jni-1.5.6-411923576037694246901.dylib' (load command 0x80000034 is unknown)
no zstd-jni-1.5.6-4 in java.library.path: /[user_directory]/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
Unsupported OS/arch, cannot find /darwin/x86_64/libzstd-jni-1.5.6-4.dylib or load zstd-jni-1.5.6-4 from system libraries. Please try building from source the jar or providing libzstd-jni-1.5.6-4 in your system.
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
    at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
    at java.base/java.lang.System.loadLibrary(System.java:1993)
    at com.github.luben.zstd.util.Native$1.run(Native.java:71)
    at com.github.luben.zstd.util.Native$1.run(Native.java:69)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
    at com.github.luben.zstd.util.Native.loadLibrary(Native.java:69)
    at com.github.luben.zstd.util.Native.load(Native.java:161)
    at com.github.luben.zstd.util.Native.load(Native.java:89)
    at com.github.luben.zstd.ZstdCompressCtx.<clinit>(ZstdCompressCtx.java:12)
...

For comparison I tested on a M2 macbook pro with the newest macOS version, which worked fine.

luben commented 1 month ago

Here is another attempt: https://github.com/luben/zstd-jni/commit/1a83dec131ba429d6804bddb2db3e7e7fa777057 . Also use older xcode that specifically comes with SDK 10.14

luben commented 1 month ago

No luck - we cannot get that older xcode:

Run maxim-lobanov/setup-xcode@v1
Switching Xcode to version '10.3'...
Available versions:
┌─────────┬──────────┬─────────────┬─────────────┬────────┬──────────────────────────────────┐
│ (index) │ version  │ buildNumber │ releaseType │ stable │ path                             │
├─────────┼──────────┼─────────────┼─────────────┼────────┼──────────────────────────────────┤
│ 0       │ '15.2.0' │ '15C500b'   │ 'GM'        │ true   │ '/Applications/Xcode_15.2.app'   │
│ 1       │ '15.1.0' │ '15C65'     │ 'GM'        │ true   │ '/Applications/Xcode_15.1.app'   │
│ 2       │ '15.0.1' │ '15A507'    │ 'GM'        │ true   │ '/Applications/Xcode_15.0.1.app' │
│ 3       │ '14.3.1' │ '14E300c'   │ 'GM'        │ true   │ '/Applications/Xcode_14.3.1.app' │
│ 4       │ '14.2.0' │ '14C18'     │ 'GM'        │ true   │ '/Applications/Xcode_14.2.app'   │
│ 5       │ '14.1.0' │ '14B47b'    │ 'GM'        │ true   │ '/Applications/Xcode_14.1.app'   │
└─────────┴──────────┴─────────────┴─────────────┴────────┴──────────────────────────────────┘
Error: Could not find Xcode version that satisfied version spec: '10.3'
chris9182 commented 1 month ago

Could you try using the 14.1.0 or even 14.3.1 one? I think this could already be good enough.

EDIT: oh i misread this. I for some reason read 10.14...

luben commented 1 month ago

I did a few unsuccessful attempts (trying to use the 10.14 SDK or compatible xcode) and just nuked them as the build was failing due to infra misses.