I was trying to suppress numeral tokens using whisper-large-v3 model and TensorRT backend, but after changing asr_options["suppress_tokens"] model output was the same. I've looked at the function call stack and their signatures and figured out some strange things on the tensorrt_llm side. For example, tensorrt_llm.GenerationSession.decode takes both SamplingConfig that has bad_words_list attribute and bad_words_list as arguments. In any case, everything points to the fact that the asr_options["suppress_tokens"] argument is ignored under the TRT backend. Am I right or could anyone help me?
I was trying to suppress numeral tokens using whisper-large-v3 model and TensorRT backend, but after changing
asr_options["suppress_tokens"]model output was the same. I've looked at the function call stack and their signatures and figured out some strange things on the tensorrt_llm side. For example,tensorrt_llm.GenerationSession.decodetakes bothSamplingConfigthat hasbad_words_listattribute andbad_words_listas arguments. In any case, everything points to the fact that theasr_options["suppress_tokens"]argument is ignored under the TRT backend. Am I right or could anyone help me?