search

huggingface / huggingface_hub

The official Python client for the Huggingface Hub.
https://huggingface.co/docs/huggingface_hub
Apache License 2.0
2.12k stars 556 forks source link

Download one GGUF model rather than all kinds of quantized models #2454

Closed kerthcet closed 3 months ago

kerthcet commented 3 months ago

Is your feature request related to a problem? Please describe.

We can use huggingface-cli download Qwen/Qwen2-0.5B-Instruct-GGUF qwen2-0_5b-instruct-q5_k_m.gguf --local-dir . --local-dir-use-symlinks False to download one specified version of gguf model, it that possible with huggingface_hub sdk?

Describe the solution you'd like

When downloading GGUF models, we can specify one version rather than download the whole files, usually containing several different quantized models.

Describe alternatives you've considered

Additional context

This is following Download as you required

Wauplin commented 3 months ago

Yes that possible using hf_hub_download. Here is a guide explaining it: https://huggingface.co/docs/huggingface_hub/guides/download#download-a-single-file.

Wauplin commented 3 months ago 
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="Qwen/Qwen2-0.5B-Instruct-GGUF",
    filename="qwen2-0_5b-instruct-q5_k_m.gguf",
    local_dir=".",
)

The --local-dir-use-symlinks is now deprecated and ignored. Symlinks between cache and local dir have been completely dropped now.

kerthcet commented 3 months ago

Ah, sorry, totally misunderstanding the meaning of filename, my mistake, thanks for the responding. Close then.