search

langchain4j / langchain4j

Java version of LangChain
https://docs.langchain4j.dev
Apache License 2.0
4.58k stars 906 forks source link

[BUG] java.lang.UnsatisfiedLinkError: Native Library C:\Users\buhake.sindi\.djl.ai\tokenizers\0.15.0-0.26.0-win-x86_64\libwinpthread-1.dll already loaded in another classloader #1803

Open TheEliteGentleman opened 1 week ago

TheEliteGentleman commented 1 week ago

There's a codebase that I'm trying to run.

This part of code (found in class DocRagIngestor)

    // Used by ContentRetriever
    @Produces
    private EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();

Produces the following issue:

[INFO] Caused by: java.lang.UnsatisfiedLinkError: Native Library C:\Users\buhake.sindi\.djl.ai\tokenizers\0.15.0-0.26.0-win-x86_64\libwinpthread-1.dll already loaded in another classloader
[INFO]  at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:167)
[INFO]  at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
[INFO]  at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2399)
[INFO]  at java.base/java.lang.Runtime.load0(Runtime.java:852)
[INFO]  at java.base/java.lang.System.load(System.java:2030)
[INFO]  at ai.djl.huggingface.tokenizers.jni.LibUtils.loadLibrary(LibUtils.java:76)
[INFO]  at ai.djl.huggingface.tokenizers.jni.LibUtils.<clinit>(LibUtils.java:41)
[INFO]  at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java:173)
[INFO]  at dev.langchain4j.model.embedding.onnx.OnnxBertBiEncoder.<init>(OnnxBertBiEncoder.java:39)
[INFO]  at dev.langchain4j.model.embedding.onnx.AbstractInProcessEmbeddingModel.loadFromJar(AbstractInProcessEmbeddingModel.java:48)
[INFO]  at dev.langchain4j.model.embedding.onnx.allminilml6v2.AllMiniLmL6V2EmbeddingModel.<clinit>(AllMiniLmL6V2EmbeddingModel.java:30)
[INFO]  at io.jefrajames.booking.DocRagIngestor.<init>(DocRagIngestor.java:44)
[INFO]  at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[INFO]  at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[INFO]  at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
[INFO]  at org.jboss.weld.injection.ConstructorInjectionPoint.newInstance(ConstructorInjectionPoint.java:119)
[INFO]  ... 25 more

I tried setting Thread context classloader to System.getSystemClassLoader() but I get this:

[INFO] Caused by: java.lang.ExceptionInInitializerError
[INFO]  at io.jefrajames.booking.DocRagIngestor.<init>(DocRagIngestor.java:44)
[INFO]  at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[INFO]  at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[INFO]  at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
[INFO]  at org.jboss.weld.injection.ConstructorInjectionPoint.newInstance(ConstructorInjectionPoint.java:119)
[INFO]  ... 25 more
[INFO] Caused by: java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke "java.io.InputStream.read(byte[], int, int)" because "inputStream" is null
[INFO]  ... 34 more
[INFO] Caused by: java.lang.NullPointerException: Cannot invoke "java.io.InputStream.read(byte[], int, int)" because "inputStream" is null
[INFO]  at dev.langchain4j.model.embedding.onnx.OnnxBertBiEncoder.loadModel(OnnxBertBiEncoder.java:229)
[INFO]  at dev.langchain4j.model.embedding.onnx.OnnxBertBiEncoder.<init>(OnnxBertBiEncoder.java:37)
[INFO]  ... 32 more

How do I resolve this?

langchain4j commented 1 week ago

Hi @TheEliteGentleman, thanks a lot for reporting!

While I don't have an immediate solution for this, you might be interested in similar issues here: