UnsatisfiedLinkError with native library in managed dependencies

[sebastianpoeplau]

Thanks for making enrich-classpath!

I've recently encountered a problem when CIDER started to use it by default: when the project contains a managed dependency with a native library and enrich-classpath is instructed to obtain sources, loading the native library fails. I've pushed a minimal project that demonstrates the error here: https://github.com/sebastianpoeplau/enrich-test.

Starting the project with lein repl (using Leiningen 2.9.1 on OpenJDK 11.0.13 for Linux) and then executing the only function in the core namespace, I get an UnsatisfiedLinkError:

enrich-test.core=> (list-images)
Execution error (UnsatisfiedLinkError) at org.newsclub.net.unix.NativeLibraryLoader/initCantLoadLibraryError (NativeLibraryLoader.java:279).
Could not load native library junixsocket-native for architecture amd64-Linux

The same error occurs in CIDER (but can of course be fixed by setting cider-enrich-classpath to nil).

I've tried to debug the problem and found that removing either the managed dependency or the "sources" classifier from project.clj fixes the issue, whereas "javadoc" doesn't seem to have an impact. Comparing the classpath in the working scenario without "sources" and the broken configuration with "sources", there are just a few more JARs when "sources" are enabled:

enrich-test.core=> (set/difference cp-working cp-broken)
#{}
enrich-test.core=> (set/difference cp-broken cp-working)
#{"/home/seb/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.12.0/jackson-databind-2.12.0-sources.jar" "/home/seb/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.12.0/jackson-annotations-2.12.0-sources.jar" "/home/seb/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0-sources.jar" "/home/seb/.m2/repository/org/yaml/snakeyaml/1.26/snakeyaml-1.26-sources.jar" "/home/seb/.m2/repository/org/jetbrains/annotations/13.0/annotations-13.0-sources.jar" "/home/seb/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/1.4.10/kotlin-stdlib-1.4.10-sources.jar" "/home/seb/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.12.0/jackson-datatype-jsr310-2.12.0-sources.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-native-common/2.3.2/junixsocket-native-common-2.3.2-sources.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-common/2.3.2/junixsocket-common-2.3.2-sources.jar" "/home/seb/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.12.0/jackson-core-2.12.0-sources.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-core/2.3.2/junixsocket-core-2.3.2-sources.jar"}

They don't look suspicious to me, except maybe that "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-core/2.3.2/junixsocket-core-2.3.2-sources.jar" appears more than once.

Do you have an idea what could be wrong?

[sebastianpoeplau]

One more observation: without the managed dependency but "sources" still enabled, I have a different version of the library on the classpath and everything works.

enrich-test.core=> (set/difference cp-without-managed-dep cp-broken)
#{"/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-common/2.3.2/junixsocket-common-2.3.2.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-core/2.3.2/junixsocket-core-2.3.2.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-native-common/2.3.2/junixsocket-native-common-2.3.2.jar"}
enrich-test.core=> (set/difference cp-broken cp-without-managed-dep)
#{"/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-common/2.3.4/junixsocket-common-2.3.4.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-core/2.3.4/junixsocket-core-2.3.4.jar" "/home/seb/.m2/repository/com/kohlschutter/junixsocket/junixsocket-native-common/2.3.4/junixsocket-native-common-2.3.4.jar"}

Could it be some property of the JAR in version 2.3.4 that enrich-classpath isn't happy with?

[vemv]

Thanks much for the detailed report!

I'll take a look later today.

Cheers - V

[vemv]

My wild guess is that one of the two is happening:

• sources shouldn't be attached for native dependencies, for some reason
• sources are ok to attach, but should go last in the classpath
  • generally this is the case already, but maybe native deps have a gotcha.

I haven't worked a lot with native deps so I'll have to study this stuff carefully.

As you point out cider-enrich-classpath can be disabled, which can also be done on a per-project fashion by simply using .dir-locals.el.

Thanks for the repro as well 🍻

[sebastianpoeplau]

The curious thing, in my opinion, is that it works without the version pinning. And actually, I just tried another thing: when I set the entry in :managed-dependencies to version 2.3.2 (i.e., the version that Leiningen would normally pull), everything works!

I guess this means that there's something in the 2.3.4 source JARs specifically that breaks enrich-classpath :thinking: What do you think?

[vemv]

Hey there!

I got the hang of it already, aided by your repro which also effectively repro'ed the issue on my Mac.

https://github.com/clojure-emacs/enrich-classpath/tree/16 is building atm, will see if it's green

---

One thing that I'd recommend in all cases is having the managed-deps like this:

:managed-dependencies [[com.kohlschutter.junixsocket/junixsocket-core "2.3.4"]
                       [com.kohlschutter.junixsocket/junixsocket-common "2.3.4"]
                       [com.kohlschutter.junixsocket/junixsocket-native-common "2.3.4"]]

This ensures all versions are consistent between them. This is a good practice in all cases, regardless of enrich-classpath. It's often needed in similarly-organised projects, notably Jackson (example)

[sebastianpoeplau]

https://github.com/clojure-emacs/enrich-classpath/tree/16 is building atm, will see if it's green

Looks like it is :tada: Awesome, thanks for looking into this so quickly!

One thing that I'd recommend in all cases is having the managed-deps like this:

That's a good point, thank you. I'll fix it in our real project.

[vemv]

1.8.0 released, as included in the latest CIDER snapshot (it might take a couple hours for MELPA to pick it up)

[sebastianpoeplau]

Works flawlessly, thanks a lot!