[Feature] Repetition Penalty support

sgl-project / sglang

SGLang is a fast serving framework for large language models and vision language models.

Apache License 2.0

4.99k stars 344 forks source link

Closed leshanbog closed 1 month ago

leshanbog commented 1 month ago

Many LLM's are prone to repetition in multi-turn conversations. So, supporting a repetition_penalty parameter is crucial for quality.

No response

zhyncs commented 1 month ago

Contributions Welcome!

merrymercy commented 1 month ago

All kinds of penalties have been supported in this PR https://github.com/sgl-project/sglang/pull/973