search

abetlen / llama-cpp-python

Python bindings for llama.cpp
https://llama-cpp-python.readthedocs.io
MIT License
8.13k stars 967 forks source link

Classifier-free guidance #506

Open abb128 opened 1 year ago

abb128 commented 1 year ago

Classifier-free guidance was implemented in llama.cpp in https://github.com/ggerganov/llama.cpp/pull/2135, but it doesn't look like there's a way to use it with these bindings. It seems like a useful feature, so it'd be nice to have this

abetlen commented 1 year ago

@abb128 totally, the low-level api is implemented for this but not the high-level api, I'll add it to the v0.2 roadmap