search

mdrokz / rust-llama.cpp

LLama.cpp rust bindings
https://crates.io/crates/llama_cpp_rs/
MIT License
322 stars 44 forks source link

Support for GBNF Grammars #31

Open benbot opened 9 months ago

benbot commented 9 months ago

Llama.cpp has had support for BNF style grammars for a while, but I don't see how I can use them with these bindings.

Is there a way?

If not, is there a good starting place for hooking them up? I could take a whack at trying to get it set up, but I don't know a lot about cpp <-> rust bindings

Szetty commented 8 months ago

We also need this, and I have tried to add the support for it, but it is not working as expected, so maybe I am not calling it when it should be called, so probably we need someone who knows this integration better

anttikallonentuni commented 4 months ago

I would also be interested in this. Or similar way to restrict the output to a specific format using the rust bindings.