search

huggingface / huggingface_hub

The official Python client for the Huggingface Hub.
https://huggingface.co/docs/huggingface_hub
Apache License 2.0
1.82k stars 470 forks source link

huggingface_hub dowload failed #2362

Closed Wenhai-Zhang closed 1 day ago

Wenhai-Zhang commented 3 days ago

Describe the bug

I checked the source of error "Distant resource does not have a Content-Length." The error seems to be caused by the size being None in the obtained HfFilemetadata information. Or rather, due to network connection, I have tried many times.

HfFileMetadata(commit_hash='c206aab77ae5967a069c4200ecb1858588528c9d', etag='018cdf165c9a3263645d925598e45f8f952990d4', location='https://hf-mirror.com/togethercomputer/evo-1-131k-base/resolve/c206aab77ae5967a069c4200ecb1858588528c9d/README.md', size=5166)
HfFileMetadata(commit_hash='c206aab77ae5967a069c4200ecb1858588528c9d', etag='d79c491957cf52bcae1722635ccdaade2fa79a2a', location='https://hf-mirror.com/togethercomputer/evo-1-131k-base/resolve/c206aab77ae5967a069c4200ecb1858588528c9d/engine.py', size=None)

Reproduction

from huggingface_hub import snapshot_download
snapshot_download(repo_id="togethercomputer/evo-1-131k-base", revision="1.1_fix")

Logs

Traceback (most recent call last):
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1754, in _get_metadata_or_catch_error
    raise FileMetadataError("Distant resource does not have a Content-Length.")
huggingface_hub.utils._errors.FileMetadataError: Distant resource does not have a Content-Length.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/work/wenhai/glm4tax/script/model_download.py", line 2, in <module>
    snapshot_download(repo_id="togethercomputer/evo-1-131k-base", revision="1.1_fix")
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 294, in snapshot_download
    thread_map(
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/tqdm/contrib/concurrent.py", line 69, in thread_map
    return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/tqdm/contrib/concurrent.py", line 51, in _executor_map
    return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/home/wenhai/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/wenhai/miniconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 268, in _inner_hf_hub_download
    return hf_hub_download(
           ^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/home/wenhai/miniconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1826, in _raise_on_head_call_error
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

System info

- huggingface_hub version: 0.23.4
- Platform: Linux-4.18.0-477.10.1.el8_8.x86_64-x86_64-with-glibc2.28
- Python version: 3.11.9
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/wenhai/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: 
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.0.0
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.3.0
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: N/A
- aiohttp: N/A
- ENDPOINT: https://hf-mirror.com
- HF_HUB_CACHE: /home/wenhai/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/wenhai/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/wenhai/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
padeoe commented 1 day ago

Hi @Wenhai-Zhang The problem you encountered is actually due to a bug on the hf-mirror site, not on huggingface.co. As the site maintainer of hf-mirror, I apologize for any inconvenience caused. This bug was fixed 3 days ago. For more details, you can refer to this issue.

Could you please check if the problem still persists for you?

Wenhai-Zhang commented 1 day ago

Thanks. I have successfully downloaded the data. I have no problem.