I already have the model stored in my network volume, but I guess the worker checks Huggingface anyway? It's looping on a gateway error.
"message":"huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/TheBloke/Nous-Capybara-34B-GPTQ"
I already have the model stored in my network volume, but I guess the worker checks Huggingface anyway? It's looping on a gateway error.
"message":"huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/TheBloke/Nous-Capybara-34B-GPTQ"
Edit: Using worker-vllm:0.2.3