add the parameter "tokenizer_mode" when using the vllm.LLM() function

allenai / open-instruct

Apache License 2.0

1.08k stars 140 forks source link

add the parameter "tokenizer_mode" when using the vllm.LLM() function #144

Closed ShuheWang1998 closed 2 months ago

ShuheWang1998 commented 2 months ago

Some tokens have different "tokenized ids" in fast and slow modes, so we need to specify the parameter "tokenizer_mode" while loading the model with vllm.