Download additional files for build

runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.

MIT License

213 stars 82 forks source link

Closed casper-hansen closed 6 months ago

casper-hansen commented 6 months ago

This downloads the full repository when building the worker-vllm image.

This has the following benefit:

You do not have to send any network request to Huggingface during initialization. This means you do not need any Huggingface token if your repository is private.