search

runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
213 stars 82 forks source link

vLLM 0.3.3 -> 0.4.0 -> 0.4.2 #62

Closed alpayariyak closed 3 months ago

joennlae commented 4 months ago

Absolut banger of a pull request :-)

xangelix commented 4 months ago

Is this compatible/ready for testing with runpod right now? Would be glad to help test.

nerdylive123 commented 3 months ago

why is this not merged yet 😊😊