search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.76k stars 3.92k forks source link

It seems that SamplingParams doesnt support the bad_words_ids parameter when generating #986

Open mengban opened 1 year ago

mengban commented 1 year ago

bad_words_ids described here is useful for production applications. However It seems that vlllm doesnt support the bad_words_ids parameter when generating. Is there a plan to support it?

  bad_words_ids(`List[List[int]]`, *optional*):
  List of list of token ids that are not allowed to be generated. Check
  [`~generation.NoBadWordsLogitsProcessor`] for further documentation and examples.
David-Lee-1990 commented 9 months ago

any comment?

whitepapercg commented 7 months ago

Up

Alvant commented 2 months ago

I hope this can be a possible bad_words_ids stuff implementation: https://github.com/vllm-project/vllm/pull/5986.