search

kherud / java-llama.cpp

Java Bindings for llama.cpp - A Port of Facebook's LLaMA model in C/C++
MIT License
279 stars 28 forks source link

where is default llama and jllama dlls? #44

Closed siddhsql closed 7 months ago

siddhsql commented 7 months ago

Hello, I am just trying to understand this project. AFAIU, there are two key dlls this project relies on to interop with llama.cpp. These are llama.dll and jllama.dll. Further, the Maven artifact comes with pre-compiled dlls for the 3 platforms: Windows, Mac and Linux. I can see that over here e.g.:

> tar -xvf ~/.m2/repository/de/kherud/llama/2.2.1/llama-2.2.1.jar
x META-INF/
x META-INF/MANIFEST.MF
x de/
x de/kherud/
x de/kherud/llama/
x de/kherud/llama/Windows/
x de/kherud/llama/Windows/x86_64/
x de/kherud/llama/Linux/
x de/kherud/llama/Linux/aarch64/
x de/kherud/llama/Linux/x86_64/
x de/kherud/llama/Mac/
x de/kherud/llama/Mac/aarch64/
x de/kherud/llama/Mac/x86_64/
x de/kherud/llama/LlamaModel$Output.class
x de/kherud/llama/OSInfo.class
x de/kherud/llama/Windows/x86_64/jllama.dll
x de/kherud/llama/Windows/x86_64/llama.dll
x de/kherud/llama/LlamaModel$1.class
x de/kherud/llama/LlamaModel$LlamaIterator.class
x de/kherud/llama/InferenceParameters$1.class
x de/kherud/llama/InferenceParameters$MiroStat.class
x de/kherud/llama/LogLevel.class
x de/kherud/llama/ProcessRunner.class
x de/kherud/llama/ModelParameters$Builder.class
x de/kherud/llama/Linux/aarch64/libllama.so
x de/kherud/llama/Linux/x86_64/libllama.so
x de/kherud/llama/Linux/x86_64/libjllama.so
x de/kherud/llama/InferenceParameters.class
x de/kherud/llama/LlamaLoader.class
x de/kherud/llama/LlamaModel.class
x de/kherud/llama/InferenceParameters$Builder.class
x de/kherud/llama/ModelParameters$1.class
x de/kherud/llama/ModelParameters.class
x de/kherud/llama/LlamaException.class
x de/kherud/llama/Mac/aarch64/libllama.dylib
x de/kherud/llama/Mac/aarch64/libjllama.dylib
x de/kherud/llama/Mac/aarch64/ggml-metal.metal
x de/kherud/llama/Mac/x86_64/libllama.dylib
x de/kherud/llama/Mac/x86_64/libjllama.dylib
x de/kherud/llama/Mac/x86_64/ggml-metal.metal
x META-INF/maven/
x META-INF/maven/de.kherud/
x META-INF/maven/de.kherud/llama/
x META-INF/maven/de.kherud/llama/pom.xml
x META-INF/maven/de.kherud/llama/pom.properties

My question is that it seems these dlls are built just-in-time when a release is published to Maven. Where is the code that does that? I can't seem to find it. thanks.

kherud commented 7 months ago

Hey @siddhsql you're right about the dlls. They are built using GitHub actions. The corresponding workflow is over here https://github.com/kherud/java-llama.cpp/blob/master/.github/workflows/release.yaml

siddhsql commented 7 months ago

thanks. this is very helpful for me to understand how it works.

On Wed, Jan 31, 2024 at 12:02 AM Konstantin Herud @.***> wrote:

Hey @siddhsql https://github.com/siddhsql you're right about the dlls. They are built using GitHub actions. The corresponding workflow is over here https://github.com/kherud/java-llama.cpp/blob/master/.github/workflows/release.yaml

— Reply to this email directly, view it on GitHub https://github.com/kherud/java-llama.cpp/issues/44#issuecomment-1918579762, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6NWEK2PJEFRWZJNLUWF4B3YRH3BBAVCNFSM6AAAAABCSEPA5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJYGU3TSNZWGI . You are receiving this because you were mentioned.Message ID: @.***>