search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 490 forks source link

Prepull model data on private swarm #541

Closed wolfganghuse closed 7 months ago

wolfganghuse commented 7 months ago

Hello, is it possible to pull all data for a model on a private swarm node ? To reduce download-traffic i would like to build pre-populated images.

Regards, Wolfgang

poedator commented 7 months ago

Hi, @wolfganghuse, you can use snapshot_download:

from huggingface_hub import snapshot_download
snapshot_download(repo_id='petals-team/StableBeluga2')

The model files will appear in your standard HF cache location.

more details in HF Hub docs: https://huggingface.co/docs/huggingface_hub/en/guides/download

wolfganghuse commented 7 months ago

Thanks @poedator I will try as soon as possible!

wolfganghuse commented 7 months ago

Just to clarify: I would download into petals CACHE-DIR and petals would pick up the model and creates the block-links accordingly?

poedator commented 7 months ago

Just to clarify: I would download into petals CACHE-DIR and petals would pick up the model and creates the block-links accordingly?

Not exactly: It will download into HF cache dir, you have to move the cached model to the Petals cache manually. Pls try and see if it works.

wolfganghuse commented 7 months ago

@poedator works exactly like you described...Thanks!