Open Abdulhanan535 opened 1 month ago
Might be interesting to add. I currently don't have the bandwidth to tackle this. I may have the time after the 0.5.4 release, but if someone else wants to take over for now, it should be quite easy:
Add the parameters related to the DRY sampler here: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/common/sampling_params.py
Define the sampler's metadata here. e.g. the conditions for triggering it (could be a related sampler param being set to higher than 0, etc) and create the necessary tensors. Plenty of examples here so you won't be lost: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/modeling/sampling_metadata.py
Finally, implement the actual sampler logic here: https://github.com/PygmalionAI/aphrodite-engine/blob/rc_054/aphrodite/modeling/layers/sampler.py
Should be plenty of examples at each step from other samplers. Please make sure you contribute to the rc_054 branch, unless you're reading this in a few weeks and that branch is merged into main.
any chance for this?
like this ``
🚀 The feature, motivation and pitch
Majority of the bots repeat sentences and due to that, it's annoying.
Implementing DRY sampler for Aphrodite Engine.
DRY is a sampler that forces the model to not repeat while mantaining the quality or even improving it, some of the backends have already support for this.
Alternatives
No response
Additional context
No response