runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
213 stars 82 forks source link

Serverless generator can not handle errors properly #61

Closed dendik closed 5 months ago