search

EricLBuehler / mistral.rs

Blazingly fast LLM inference.
MIT License
3.37k stars 242 forks source link

Add DRY repetition penalty #635

Closed p-e-w closed 1 week ago

p-e-w commented 1 month ago

DRY is a modern repetition penalty that is superior to the standard frequency and presence penalties at preventing repetition, while having virtually none of their negative effects on language quality.

Original implementation with detailed explanation: https://github.com/oobabooga/text-generation-webui/pull/5677

C++ implementation in llama.cpp (pending merge): https://github.com/ggerganov/llama.cpp/pull/6839

Independent C++ implementation in Kobold (merged): https://github.com/LostRuins/koboldcpp/pull/982

EricLBuehler commented 1 month ago

@p-e-w I just added a PR for this. If you could give it a review that would be amazing.