runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
238 stars 96 forks source link

10x Faster New Worker #18

Closed alpayariyak closed 10 months ago