ThiloteE
commented
2 weeks ago Solutions:
-
A) Have no CUDA installed on your system (and have no CUDA paths set in your systems Environment Variables), then it should automatically detect the correct location of the CUDA path that is managed by JabRef (and DeepJavaLibrary (djl) and the system will also somehow find all dependencies.
-
B) (If you have multiple CUDA installed on your system) Add
C:\Users\USER\.djl.ai\pytorch\2.4.0-cu124-win-x86_64to "PATH" in Windows 10 Environment Variables. Can be done via searching for environment variables and manually adding the folder path. See https://www.howtogeek.com/787217/how-to-edit-environment-variables-on-windows-10-or-11/
Explanation about what happened to me:
My hypothesis about what happened: Since I had multiple CUDAs installed on my system and I had not set the PATH and System Environment Variables, the embedding model was not functioning and only the LLM was fully functional, which made it seem like embeddings were not sent to GPT4All, while in reality, no embeddings had ever been created in the first place. I confirmed LLMs being functional, while testing local API servers like GPT4All, Ollama or llama.cpp, as reported in issue https://github.com/JabRef/jabref/issues/12114
I also had lots of x86 Microsoft Visual C++ Redistributables installed, which are not needed on my x64 system and that also might have caused some conflicts, but the main issue was the path issue, which caused the embeddings model to not function.
Links and comments that helped me find the answers:
- https://stackoverflow.com/questions/6092200/how-to-fix-an-unsatisfiedlinkerror-cant-find-dependent-libraries-in-a-jni-pro
- https://github.com/KernFerm/nvidia-installation-guide?tab=readme-ov-file#setting-up-cuda-118-with-cudnn-on-windows
- https://forum.image.sc/t/trouble-getting-gpu-to-work-with-instanseg-qupath/102042/22
- https://forum.image.sc/t/trouble-getting-gpu-to-work-with-instanseg-qupath/102042/26
JabRef commit ed0e90e872fd992a9cae6596e6d74836233bf105
I am on Windows 10 with an Nvidia Geforce GTX 1060 3GB and an AMD Ryzen 5 5600 that supports AVX and AVX2.
Error message:
This was a Windows issue. I did install the newest Visual C++ redistributable from Microsoft and restarted and I also deleted the
.djl.aifolder in my home directory, as adviced. Both of which did not solve the issue.I then used Dependency Walker on torch_cuda.dll. It found that lots of dependencies were not satisfied, but dependency walker is a little outdated. See https://stackoverflow.com/a/36244483/873282. One should use https://github.com/lhak/Dependencies IMHO.
Funny enough, this is a regression introduced by commit c280eb79da587249fa7ec477a358952baaed5bae (https://github.com/JabRef/jabref/pull/11773). That one upgrades ai.djl:pytorch:pytorch-model-zoo from 0.29 to 0.30, which comes with different pytorch cuda dlls. Version 0.29.0 still works. Version 0.30.0 does not.
JabRef documenation:
DJL Documentation:
DJL related issues: