Closed Sapessii closed 6 months ago
Hi,
running runpod/worker-vllm:0.3.0-cuda12.1.0 in serverless and setting the env variable MODEL_REVISION it's not read and it keeps downloading the main branch.
Any suggestion? Thank you
solved, the env is MODEL_NAME_REVISION, not MODEL_REVISION
Just fixed this to use MODEL_REVISION and TOKENIZER_REVISION, thanks for pointing it out!
MODEL_REVISION
TOKENIZER_REVISION
Hi,
running runpod/worker-vllm:0.3.0-cuda12.1.0 in serverless and setting the env variable MODEL_REVISION it's not read and it keeps downloading the main branch.
Any suggestion? Thank you