Issue with llamacpp version larger than 0.2.37

noamgat / lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model

MIT License

1.42k stars 65 forks source link

Issue with llamacpp version larger than 0.2.37 #72

Closed isamu-isozaki closed 7 months ago

isamu-isozaki commented 7 months ago

@noamgat I haven't fully looked into this but on version 0.2.38 of llamacpp with this library and outlines as well, the generation fails. The change to llamacpp 0.2.37 does fix both libraries' generation problems while both does struggle with 0.2.38. The outlines devs seem to have a fix here

michael-newsrx commented 7 months ago

According to the llama-cpp-python changelog, they fixed an off-by-one error in version 0.2.42.

https://github.com/abetlen/llama-cpp-python/blob/main/CHANGELOG.md

isamu-isozaki commented 7 months ago

Oh awesome I'll test out

noamgat commented 7 months ago

Verified that new version is working. Closing.