llama: add grammar-based sampling

alpaca-core / ac-local

Alpaca Core local inference SDK

MIT License

1 stars 0 forks source link

llama: add grammar-based sampling #14

Open iboB opened 3 months ago

iboB commented 3 months ago

Intentionally skipped while implementing #3

iboB commented 1 month ago

Thanks to the latest update of llama.cpp we already have this as a feature. It should work, but there are still no examples or tests. There is no api integration