Bug: UnsatisfiedLinkError Loading Native Library for Java

[ccjernigan]

Describe the bug

Touching Index causes the static initializer for loading the native library to execute. This doesn't seem to be working correctly.

Results:

> Task :run FAILED
Hello World!
Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/4l/920gn80x5yj5fw3qk0nh62kh0000gn/T/nativeutils117144147798541/libusearch.so: dlopen(/private/var/folders/4l/920gn80x5yj5fw3qk0nh62kh0000gn/T/nativeutils117144147798541/libusearch.so, 0x0001): tried: '/private/var/folders/4l/920gn80x5yj5fw3qk0nh62kh0000gn/T/nativeutils117144147798541/libusearch.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/4l/920gn80x5yj5fw3qk0nh62kh0000gn/T/nativeutils117144147798541/libusearch.so' (no such file), '/private/var/folders/4l/920gn80x5yj5fw3qk0nh62kh0000gn/T/nativeutils117144147798541/libusearch.so' (not a mach-o file)
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2394)
        at java.base/java.lang.Runtime.load0(Runtime.java:755)
        at java.base/java.lang.System.load(System.java:1953)
        at cloud.unum.usearch.NativeUtils.loadLibraryFromJar(NativeUtils.java:110)
        at cloud.unum.usearch.Index.<clinit>(Index.java:295)
        at cloud.unum.usearch.Index$Config.build(Index.java:200)
        at MainKt.main(Main.kt:6)

usearch.zip

Steps to reproduce

Using the attached

1. ./gradlew run

Expected behavior

Application runs

USearch version

2.8.12

Operating System

macOS Sonoma 14.1.1

Hardware architecture

Arm

Which interface are you using?

Other bindings

Contact Details

No response

Is there an existing issue for this?

• [X] I have searched the existing issues

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[ashvardanian]

Hey @ccjernigan! Thanks for spotting the issue! I am very unexperienced in packaging JNI libraries. Together with @superkelvint we've been trying to fix those, but the iteration cycle is super slow. Are you familiar with the process? Can you help us patch it?

[ccjernigan]

I'm not a JNI expert, although maybe we can figure this out together.

How are you iterating currently? I like to publish/consume libraries from Maven local (~/.m2) to help troubleshoot with packaging of libraries.

It also looks like this issue may be platform-specific. With a GitHub Actions workflow, it works on Linux but not Windows or macOS. For the macOS issue, I confirmed the problem occurs on Intel and M1 so it isn't a simple architecture issue either.

See https://github.com/ccjernigan/search/actions/runs/6907563003

There's only a single .so file packaged into the JAR. Shouldn't there be either multiple JAR files (on per platform) or multiple .so files within the JAR file? Basically, don't we need a matrix of native libraries for OS (Windows, macOS, and Linux) and architecture (ARM and x86)?

[ashvardanian]

Yes, @ccjernigan, we need a matrix of JARs, but I am not sure about how to configure Bazel and GitHub CI for that output.

I am currently overwhelmed with clustering and dimensionality reduction functionality and the upcoming v3 release, so if you could contribute patches, that would help us a lot!

[patedhav]

we are also looking for the same fix while using to usearch + java binding run on the M1 mac.

[ashvardanian]

I believe #407 may be fixing this issue, @patedhav and @ccjernigan 🤗 Thanks to @mccullocht 🙌 I'm planning to merge into main once I understand where the root cause of replacements bug comes from.

[patedhav]

@ashvardanian @mccullocht #407 seems to be Index.get(int key) throwing exception issue. Does it also include building jar with MacOS M1 .so1 file?

[ashvardanian]

@patedhav seems so, as @mccullocht has added a Clang build and fixed library name resolution to fetch .dylib on MacOS. Correct me if I'm wrong 🤗

[patedhav]

@ashvardanian thanks for the update, looking forward to get it to main and java jar release!