search

kherud / java-llama.cpp

Java Bindings for llama.cpp - A Port of Facebook's LLaMA model in C/C++
MIT License
279 stars 28 forks source link

Upgrade llama.cpp to b3265, support gemma2, remove beam parameter[ https://github.com/ggerganov/llama.cpp/pull/7985 ] #71

Closed zdl010 closed 1 month ago

zdl010 commented 2 months ago

Upgrade llama.cpp to b3265, support gemma2, remove beam parameter[ https://github.com/ggerganov/llama.cpp/pull/7985 ]

kherud commented 2 months ago

Hey @zdl010 thanks for the pull request! I'll update the C++ code to b3265 and merge after that.

ardinursyamsu commented 1 month ago

@kherud I see there's a lot of update in server.cpp in llama.cpp. Is that would take long to adjust in server.hpp?

kherud commented 1 month ago

Hey @ardinursyamsu yeah it's a challenge to keep up with the rapid development of llama.cpp. Sometimes there are bugs, where it isn't obvious if they come from llama.cpp or the Java binding. I'll have another try to update to the latest version later today, though.

kherud commented 1 month ago

Ok, there was a change in llama.cpp to no longer statically link the ggml library (see https://github.com/ggerganov/llama.cpp/issues/8166), which caused the previous Windows builds here to fail. Not sure why it worked for Linux/MacOS (probably because of rpath). I'll look for a solution tomorrow and release a new version then.