[Feature] DRY repetition penalty

vnkc1 commented 2 weeks ago

[X] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[X] 2. Please use English, otherwise it will be closed.

DRY is a modern repetition penalty which ramps up penalties on N-grams to avoid looping behavior.

The penalty is commonly used on Llama 3.1 8B by practitioners to avoid repetition spirals.

zhyncs commented 2 weeks ago

vnkc1 commented 2 weeks ago

Thanks

vnkc1 commented 2 weeks ago

Closing this issue

sgl-project / sglang