Fix the forwarding of the length penalty parameter

IBM / text-generation-inference

IBM development fork of https://github.com/huggingface/text-generation-inference

Apache License 2.0

57 stars 30 forks source link

Fix the forwarding of the length penalty parameter #70

Closed maxdebayser closed 7 months ago

maxdebayser commented 7 months ago

It was erroneously tied to the presence of the repetition penalty parameter. So when the repetition penalty wasn't set, the server would get this parameter even though the client sent it to the router.