Closed Lhemamou closed 2 months ago
+1
+1, looking to figure this out soon
Same issue here. There is a blocking bug on Llama3 that has been fixed with v0.4.1.
Pretty please
Hi all, thank you for raising this issue! I have just merged the vLLM 0.4.2 update into main, you can use it by changing your Docker image in your endpoint from runpod/worker-vllm:stable-cudaX.X.X
to runpod/worker-vllm:dev-cudaX.X.X
. From my testing thus far, everything seems in order, but if you notice any issues, please let me know. After an initial test period, I'll release the update officially to replace the default stable images. Thanks all!
merged into stable release
Hello everyone,
I would like to update the vLLM version to v0.4.1 in order to get access to LLAMA3 but i don't know how modify the fork runpod/vllm-fork-for-sls-worker. Could you please guide me ? Happy to help in some way!