JabRef / jabref

Graphical Java application for managing BibTeX and biblatex (.bib) databases
https://devdocs.jabref.org
MIT License
3.63k stars 2.59k forks source link

AI: Unable to chat. And error occured while building the embedding model. Failed to load PyTorch native library. #12169

Closed ThiloteE closed 2 days ago

ThiloteE commented 2 days ago

JabRef commit ed0e90e872fd992a9cae6596e6d74836233bf105

I am on Windows 10 with an Nvidia Geforce GTX 1060 3GB and an AMD Ryzen 5 5600 that supports AVX and AVX2. image

Error message:

C:\Prog\Development\JabRef_ThiloteE-JabRef\ThiloteE-JabRef\jabref>gradle run

> Configure project :
Project : => 'org.jabref' Java module

> Task :run
WARNING: Using incubator modules: jdk.incubator.vector
Messages are not initialized before accessing key: Display help on command line options
2024-09-18 02:31:21 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.addStylesheetToWatchlist()
INFO: Watching css C:\Prog\Development\JabRef_ThiloteE-JabRef\ThiloteE-JabRef\jabref\build\resources\main\org\jabref\gui\Base.css for live updates
2024-09-18 02:31:21 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Not updating theme because it hasn't changed
2024-09-18 02:31:21 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Theme set to Theme{type=DEFAULT, name=''} with base css StyleSheet{file:/C:/Prog/Development/JabRef_ThiloteE-JabRef/ThiloteE-JabRef/jabref/build/resources/main/org/jabref/gui/Base.css}
2024-09-18 02:31:21 [pool-2-thread-1] org.jabref.logic.ai.ingestion.model.UpdateEmbeddingModelTask.call()
INFO: Downloading embedding model...
2024-09-18 02:31:22 [JavaFX Application Thread] org.jabref.logic.push.CitationCommandString.from()
INFO: Wrong indexes -1 -1 for completeCiteCommand \parencite. Using default delimiter and suffix.
2024-09-18 02:31:23 [JavaFX Application Thread] org.jabref.gui.StateManager.setActiveDatabase()
INFO: No open database detected
2024-09-18 02:31:25 [pool-2-thread-2] org.jabref.gui.JabRefDialogService.notify()
INFO: Opening: 'C:\server-t-150\Jabref & LatexTests\OneMillionEntries.bib'
2024-09-18 02:31:25 [pool-2-thread-2] org.jabref.gui.JabRefDialogService.notify()
INFO: Opening: 'C:\server-t-150\Jabref & LatexTests\test-Files.bib'
2024-09-18 02:31:25 [pool-2-thread-3] org.jabref.gui.JabRefDialogService.notify()
INFO: Opening: 'C:\server-t-150\Jabref & LatexTests\Chocolate.bib'
2024-09-18 02:31:26 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-09-18 02:31:27 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/asmjit.dll.gz ...
2024-09-18 02:31:27 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/c10_cuda.dll.gz ...
2024-09-18 02:31:28 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/c10.dll.gz ...
2024-09-18 02:31:28 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-09-18 02:31:28 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/caffe2_nvrtc.dll.gz ...
2024-09-18 02:31:28 [JavaFX Application Thread] org.apache.lucene.internal.vectorization.PanamaVectorizationProvider.<init>()
INFO: Java vector incubator API enabled; uses preferredBitSize=256; FMA enabled
2024-09-18 02:31:29 [JavaFX Application Thread] org.apache.lucene.store.MemorySegmentIndexInputProvider.<init>()
INFO: Using MemorySegmentIndexInput with Java 21 or later; to disable start with -Dorg.apache.lucene.store.MMapDirectory.enableMemorySegments=false
2024-09-18 02:31:29 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cublas64_12.dll.gz ...
2024-09-18 02:31:35 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cublasLt64_12.dll.gz ...
2024-09-18 02:32:02 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudart64_12.dll.gz ...
2024-09-18 02:32:03 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_adv64_9.dll.gz ...
2024-09-18 02:32:17 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_cnn64_9.dll.gz ...
2024-09-18 02:32:19 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_engines_precompiled64_9.dll.gz ...
2024-09-18 02:32:51 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_engines_runtime_compiled64_9.dll.gz ...
2024-09-18 02:32:52 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_graph64_9.dll.gz ...
2024-09-18 02:32:53 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_heuristic64_9.dll.gz ...
2024-09-18 02:32:59 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn_ops64_9.dll.gz ...
2024-09-18 02:33:06 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cudnn64_9.dll.gz ...
2024-09-18 02:33:06 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cufft64_11.dll.gz ...
2024-09-18 02:33:24 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cufftw64_11.dll.gz ...
2024-09-18 02:33:24 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cupti64_2024.1.0.dll.gz ...
2024-09-18 02:33:25 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/curand64_10.dll.gz ...
2024-09-18 02:33:31 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cusolver64_11.dll.gz ...
2024-09-18 02:33:37 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cusolverMg64_11.dll.gz ...
2024-09-18 02:33:42 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/cusparse64_12.dll.gz ...
2024-09-18 02:33:59 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/fbgemm.dll.gz ...
2024-09-18 02:34:00 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/libiomp5md.dll.gz ...
2024-09-18 02:34:01 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/libiompstubs5md.dll.gz ...
2024-09-18 02:34:01 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/nvJitLink_120_0.dll.gz ...
2024-09-18 02:34:04 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/nvrtc-builtins64_124.dll.gz ...
2024-09-18 02:34:04 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/nvrtc64_120_0.dll.gz ...
2024-09-18 02:34:07 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/nvToolsExt64_1.dll.gz ...
2024-09-18 02:34:07 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/torch_cpu.dll.gz ...
2024-09-18 02:34:13 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/torch_cuda.dll.gz ...
2024-09-18 02:35:02 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/torch.dll.gz ...
2024-09-18 02:35:03 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/uv.dll.gz ...
2024-09-18 02:35:03 [pool-2-thread-1] ai.djl.pytorch.jni.LibUtils.downloadPyTorch()
INFO: Downloading https://publish.djl.ai/pytorch/2.4.0/cu124/win-x86_64/native/lib/zlibwapi.dll.gz ...
2024-09-18 02:35:06 [JavaFX Application Thread] org.jabref.logic.ai.ingestion.model.JabRefEmbeddingModel.lambda$startRebuildingTask$1()
ERROR: An error occurred while building the embedding model: ai.djl.engine.EngineException: Failed to load PyTorch native library
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:90)
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:41)
        at ai.djl.api@0.30.0/ai.djl.engine.Engine.getEngine(Engine.java:190)
        at ai.djl.api@0.30.0/ai.djl.Model.newInstance(Model.java:99)
        at ai.djl.api@0.30.0/ai.djl.repository.zoo.BaseModelLoader.createModel(BaseModelLoader.java:224)
        at ai.djl.api@0.30.0/ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:165)
2024-09-18 02:35:06 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: An error occurred while building the embedding model
        at ai.djl.api@0.30.0/ai.djl.repository.zoo.Criteria.loadModel(Criteria.java:151)
        at org.jabref@100.0.0/org.jabref.logic.ai.ingestion.model.DeepJavaEmbeddingModel.<init>(DeepJavaEmbeddingModel.java:23)
        at org.jabref@100.0.0/org.jabref.logic.ai.ingestion.model.UpdateEmbeddingModelTask.call(UpdateEmbeddingModelTask.java:65)
        at org.jabref@100.0.0/org.jabref.logic.ai.ingestion.model.UpdateEmbeddingModelTask.call(UpdateEmbeddingModelTask.java:20)
        at org.jabref@100.0.0/org.jabref.gui.util.UiTaskExecutor$1.call(UiTaskExecutor.java:188)
        at javafx.graphics@23/javafx.concurrent.Task$TaskCallable.call(Task.java:1401)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.UnsatisfiedLinkError: C:\Users\Thilo\.djl.ai\pytorch\2.4.0-cu124-win-x86_64\torch_cuda.dll: Can't find dependent libraries
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2418)
        at java.base/java.lang.Runtime.load0(Runtime.java:852)
        at java.base/java.lang.System.load(System.java:2025)
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.jni.LibUtils.loadNativeLibrary(LibUtils.java:379)
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.jni.LibUtils.loadLibTorch(LibUtils.java:195)
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:82)
        at ai.djl.pytorch_engine@0.30.0/ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:53)
        ... 17 more

This was a Windows issue. I did install the newest Visual C++ redistributable from Microsoft and restarted and I also deleted the .djl.ai folder in my home directory, as adviced. Both of which did not solve the issue.

I then used Dependency Walker on torch_cuda.dll. It found that lots of dependencies were not satisfied, but dependency walker is a little outdated. See https://stackoverflow.com/a/36244483/873282. One should use https://github.com/lhak/Dependencies IMHO.

Funny enough, this is a regression introduced by commit c280eb79da587249fa7ec477a358952baaed5bae (https://github.com/JabRef/jabref/pull/11773). That one upgrades ai.djl:pytorch:pytorch-model-zoo from 0.29 to 0.30, which comes with different pytorch cuda dlls. Version 0.29.0 still works. Version 0.30.0 does not.

JabRef documenation:

DJL Documentation:

DJL related issues:

ThiloteE commented 2 days ago

Solutions:

Explanation about what happened to me:

My hypothesis about what happened: Since I had multiple CUDAs installed on my system and I had not set the PATH and System Environment Variables, the embedding model was not functioning and only the LLM was fully functional, which made it seem like embeddings were not sent to GPT4All, while in reality, no embeddings had ever been created in the first place. I confirmed LLMs being functional, while testing local API servers like GPT4All, Ollama or llama.cpp, as reported in issue https://github.com/JabRef/jabref/issues/12114

I also had lots of x86 Microsoft Visual C++ Redistributables installed, which are not needed on my x64 system and that also might have caused some conflicts, but the main issue was the path issue, which caused the embeddings model to not function.

Links and comments that helped me find the answers: