Open josharian opened 7 months ago
Yes, we should do that. I'm also thinking about moving all the grammar stuff into the ggml
core library so that it becomes available everywhere. But the main problem is re-implementing the C++ stuff in C
the main problem is re-implementing the C++ stuff in C
Another option is to hide it all behind a very simple C facade, something like new
/init
, parse
, free
. (At least, that would suffice for Go bindings.)
Btw, I've noticed some crashes in the GBNF parser. I am planning to set up some fuzzing for it soon to try to shake them out.
I'd like to add Go binding support for grammars (for https://github.com/ggerganov/whisper.cpp/issues/1697). It's a bit inconvenient now, because the grammar support is off in an examples directory, and adding stuff from examples to libwhisper.a feels wrong; ditto for the header file.
GBNF seems pretty well established at this point (it is in common in llama.cpp). It'd be nice to make it easier to support.
Could we promote grammar support to core whisper.h/whisper.cpp, similar to llama.cpp, where you simply provide a grammar string?