Open cooperll opened 3 months ago
I just open a PR: https://github.com/edgenai/llama_cpp-rs/pull/91
In the meantime, if you want to try it out, just put this in the Cargo.toml:
llama_cpp = { git = "https://github.com/vargad/llama_cpp-rs.git", branch = "bump_3038" }
I only tested Phi3 4K model, that works with the change above.
This is great. Thank you.
On adding llama_cpp-rs to my Cargo.toml, llama.cpp seems to be locked to an older version. I'm trying to use Phi-3 128k in a project and I'm unable to because the PR that was merged into llama.cpp about two weeks ago.
Is there an easy way to make sure that llama_cpp-rs is using the latest llama.cpp commit?