search

kherud / java-llama.cpp

Java Bindings for llama.cpp - A Port of Facebook's LLaMA model in C/C++
MIT License
279 stars 28 forks source link

Need help: No native library found for os.name=Windows, os.arch=x86_64 #24

Closed j0schihatake closed 5 months ago

j0schihatake commented 10 months ago

Good afternoon, I tried to use your solution, imported the library into maven. llama.cpp is launched in the docker container and the build is performed there. The required libraries are indicated: Linux: libllama.so, libjllama.so found in a container in the /home/ folder, so I start the application like this: CMD java -jar -Djava.library.path=/home/ app.jar, unfortunately I don’t understand what yet for libraries, here is a link to the repository https://github.com/j0schihatake/NN_llama_cpp_docker/tree/java_llama I previously used FastApi for access, but I want to work from java. Thank you very much in advance if you can tell me what’s wrong. Here is a link to a java spring application that is added to the docker container as app.jar: https://github.com/j0schihatake/NN_Java_Llama

kherud commented 10 months ago

Hi @j0schihatake we are currently working on improving out of the box support for this Java library. If it doesn't work for you yet and you don't want to wait for updates, you can try to compile the library yourself. Please follow this description for it.

The libraries will be installed to java-llama.cpp/src/main/resources/de/kherud/llama/Windows/x86_64. You'll have to slightly modify your startup command to:

java -jar -Djava.library.path=/home/ -Dde.kherud.llama.lib.path=java-llama.cpp/src/main/resources/de/kherud/llama/Windows/x86_64 app.jar

Note that you might have to pass an absolute path instead of the relative java-llama.cpp/src/main/resources/de/kherud/llama/Windows/x86_64.

Let me know if you have any questions or problems!