Closed Palmik closed 11 months ago
Update vllm dependency to latest version, and comment out max_num_batched_tokens in handler.
max_num_batched_tokens
Thank you @Palmik
Update vllm dependency to latest version, and comment out
max_num_batched_tokens
in handler.