search

mdrokz / rust-llama.cpp

LLama.cpp rust bindings
https://crates.io/crates/llama_cpp_rs/
MIT License
322 stars 44 forks source link

Error running Phi2 Models #29

Open jasonw247 opened 9 months ago

jasonw247 commented 9 months ago

When attempting to run dolphin-2_6-phi-2.Q4_0.gguf I'm getting error loading model: unknown model architecture: 'phi2'.

Phi2 support was added a couple of weeks ago: https://github.com/ggerganov/llama.cpp/pull/4490.

Is there a way to include this?

For reference, I am using this repo as part of a different package using current master:

[dependencies]
llama_cpp_rs = { git = "https://github.com/mdrokz/rust-llama.cpp.git", rev = "4922cac", features = ["metal"] }