search

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
https://lmdeploy.readthedocs.io/en/latest/
Apache License 2.0
4.56k stars 409 forks source link

stream_infer 不支持'length_penalty' #1261

Open JidongZhang-THU opened 8 months ago

JidongZhang-THU commented 8 months ago

TypeError: stream_infer() got an unexpected keyword argument 'length_penalty' 请问stream infer 不支持length_penalty 吗？

zhyncs commented 8 months ago

TypeError: stream_infer() got an unexpected keyword argument 'length_penalty' 请问stream infer 不支持length_penalty 吗？

Currently not supported.

https://huggingface.co/docs/transformers/en/main_classes/text_generation

length_penalty (float, optional, defaults to 1.0) — Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), length_penalty > 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences.