Setting minP allows for better results even at higer temperatures by rejecting tokens that are too unlikely. It is now supported in llama.cpp, and it makes both topP and topK almost superfluous.
The Solution
It should be done the same way as you did for maxP, I guess.
Considered Alternatives
Don't do it if you don't want to.
Additional Context
No response
Related Features to This Feature Request
[ ] Metal support
[ ] CUDA support
[ ] Grammar
Are you willing to resolve this issue by submitting a Pull Request?
No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.
Feature Description
Setting minP allows for better results even at higer temperatures by rejecting tokens that are too unlikely. It is now supported in llama.cpp, and it makes both topP and topK almost superfluous.
The Solution
It should be done the same way as you did for maxP, I guess.
Considered Alternatives
Don't do it if you don't want to.
Additional Context
No response
Related Features to This Feature Request
Are you willing to resolve this issue by submitting a Pull Request?
No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.