Closed zhyncs closed 2 months ago
ref vLLM https://docs.vllm.ai/en/latest/dev/sampling_params.html
ignore_eos – Whether to ignore the EOS token and continue generating tokens after the EOS token is generated.
LMDeploy https://lmdeploy.readthedocs.io/en/latest/api/pipeline.html?highlight=ignore_eos
ignore_eos (bool) – Indicator to ignore the eos_token_id or not
DeepSpeed-MII https://github.com/microsoft/DeepSpeed-MII
ignore_eos: bool (Defaults to False) Setting to True prevents generation from ending when the EOS token is encountered.
Due to the lack of response for a long time, this issue will be closed for now. It can be reopened later if necessary. Thanks.
System Info
as titled general question
Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
N/A
Expected behavior
N/A
actual behavior
N/A
additional notes
N/A