search

huggingface / huggingface_hub

The official Python client for the Huggingface Hub.
https://huggingface.co/docs/huggingface_hub
Apache License 2.0
1.83k stars 471 forks source link

Failed to download Phi-3-mini-4k-instruct to local dir #2276

Closed XiaofuHuang closed 1 month ago

XiaofuHuang commented 1 month ago

Describe the bug

Failed to download microsoft/Phi-3-mini-4k-instruct repo into local dir using huggingface cli

Reproduction

Run command: huggingface-cli download microsoft/Phi-3-mini-4k-instruct --revision main --local-dir ./model-cache/microsoft/Phi-3-mini-4k-instruct

Logs

2024-05-08T09:03:28.087428030Z 
Fetching 19 files:   0%|          | 0/19 [00:00<?, ?it/s]
Fetching 19 files:   0%|          | 0/19 [00:00<?, ?it/s]
2024-05-08T09:03:28.613384779Z Downloading 'CODE_OF_CONDUCT.md' to 'model-cache/microsoft/Phi-3-mini-4k-instruct/.huggingface/download/CODE_OF_CONDUCT.md.f9ba8cf65f3e3104dd061c178066ec8247811f33.incomplete'
2024-05-08T09:03:28.658570498Z Downloading 'config.json' to 'model-cache/microsoft/Phi-3-mini-4k-instruct/.huggingface/download/config.json.a95e91d511816161e96d3ef819866e2ad98ecbcb.incomplete'
2024-05-08T09:03:28.667740985Z Downloading 'configuration_phi3.py' to 'model-cache/microsoft/Phi-3-mini-4k-instruct/.huggingface/download/configuration_phi3.py.f4553db23ac65c608fd150a14acbd04d3ff80a0f.incomplete'
2024-05-08T09:03:28.691318972Z Downloading 'NOTICE.md' to 'model-cache/microsoft/Phi-3-mini-4k-instruct/.huggingface/download/NOTICE.md.ee58e836b8cb628406447bae6b6b75a0fa553143.incomplete'
2024-05-08T09:03:28.742833151Z Download complete. Moving file to model-cache/microsoft/Phi-3-mini-4k-instruct/CODE_OF_CONDUCT.md
2024-05-08T09:03:28.792560893Z Download complete. Moving file to model-cache/microsoft/Phi-3-mini-4k-instruct/config.json
2024-05-08T09:03:28.792685557Z Download complete. Moving file to model-cache/microsoft/Phi-3-mini-4k-instruct/configuration_phi3.py
2024-05-08T09:03:28.910355037Z Download complete. Moving file to model-cache/microsoft/Phi-3-mini-4k-instruct/NOTICE.md
2024-05-08T09:03:29.354325711Z Traceback (most recent call last):
2024-05-08T09:03:29.354359704Z   File "/opt/conda/bin/huggingface-cli", line 8, in <module>
2024-05-08T09:03:29.354364954Z     sys.exit(main())
2024-05-08T09:03:29.354368601Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/commands/huggingface_cli.py", line 51, in main
2024-05-08T09:03:29.354371346Z     service.run()
2024-05-08T09:03:29.354374061Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/commands/download.py", line 146, in run
2024-05-08T09:03:29.354385222Z     print(self._download())  # Print path to downloaded files
2024-05-08T09:03:29.354389350Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/commands/download.py", line 180, in _download
2024-05-08T09:03:29.354392476Z     return snapshot_download(
2024-05-08T09:03:29.354395381Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
2024-05-08T09:03:29.354398056Z     return fn(*args, **kwargs)
2024-05-08T09:03:29.354400832Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 294, in snapshot_download
2024-05-08T09:03:29.354403486Z     thread_map(
2024-05-08T09:03:29.354406251Z   File "/opt/conda/lib/python3.10/site-packages/tqdm/contrib/concurrent.py", line 69, in thread_map
2024-05-08T09:03:29.354409127Z     return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs)
2024-05-08T09:03:29.354411782Z   File "/opt/conda/lib/python3.10/site-packages/tqdm/contrib/concurrent.py", line 51, in _executor_map
2024-05-08T09:03:29.354414888Z     return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
2024-05-08T09:03:29.354417633Z   File "/opt/conda/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
2024-05-08T09:03:29.354437871Z     for obj in iterable:
2024-05-08T09:03:29.354441357Z   File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
2024-05-08T09:03:29.354508939Z     yield _result_or_cancel(fs.pop())
2024-05-08T09:03:29.354526772Z   File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
2024-05-08T09:03:29.354576111Z     return fut.result(timeout)
2024-05-08T09:03:29.354590888Z   File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 458, in result
2024-05-08T09:03:29.354648055Z     return self.__get_result()
2024-05-08T09:03:29.354654047Z   File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2024-05-08T09:03:29.354711945Z     raise self._exception
2024-05-08T09:03:29.354716143Z   File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 58, in run
2024-05-08T09:03:29.354771136Z     result = self.fn(*self.args, **self.kwargs)
2024-05-08T09:03:29.354775665Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 268, in _inner_hf_hub_download
2024-05-08T09:03:29.354842791Z     return hf_hub_download(
2024-05-08T09:03:29.354848662Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
2024-05-08T09:03:29.354886032Z     return fn(*args, **kwargs)
2024-05-08T09:03:29.354905669Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1202, in hf_hub_download
2024-05-08T09:03:29.355130962Z     return _hf_hub_download_to_local_dir(
2024-05-08T09:03:29.355137333Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1406, in _hf_hub_download_to_local_dir
2024-05-08T09:03:29.355380309Z     paths = get_local_download_paths(local_dir=local_dir, filename=filename)
2024-05-08T09:03:29.355396799Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_local_folder.py", line 138, in get_local_download_paths
2024-05-08T09:03:29.355448688Z     metadata_path = _huggingface_dir(local_dir) / "download" / f"{sanitized_filename}.metadata"
2024-05-08T09:03:29.355468435Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_local_folder.py", line 223, in _huggingface_dir
2024-05-08T09:03:29.355538245Z     with WeakFileLock(gitignore_lock):
2024-05-08T09:03:29.355544156Z   File "/opt/conda/lib/python3.10/contextlib.py", line 135, in __enter__
2024-05-08T09:03:29.355571598Z     return next(self.gen)
2024-05-08T09:03:29.355575034Z   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_fixes.py", line 84, in WeakFileLock
2024-05-08T09:03:29.355641119Z     lock.acquire()
2024-05-08T09:03:29.355655406Z   File "/opt/conda/lib/python3.10/site-packages/filelock/_api.py", line 255, in acquire
2024-05-08T09:03:29.355719641Z     self._acquire()
2024-05-08T09:03:29.355734599Z   File "/opt/conda/lib/python3.10/site-packages/filelock/_unix.py", line 39, in _acquire
2024-05-08T09:03:29.355748344Z     fd = os.open(self.lock_file, open_flags, self._context.mode)
2024-05-08T09:03:29.355751590Z FileNotFoundError: [Errno 2] No such file or directory: 'model-cache/microsoft/Phi-3-mini-4k-instruct/.huggingface/.gitignore.lock'

System info

- huggingface_hub version: 0.23.0
- Platform: Linux-5.4.0-1127-azure-x86_64-with-glibc2.35
- Python version: 3.10.14
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /root/.cache/huggingface/token
- Has saved token ?: False
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.3.0
- Jinja2: 3.1.3
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.2.0
- hf_transfer: N/A
- gradio: 4.29.0
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.7.1
- aiohttp: 3.9.5
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /root/.cache/huggingface/hub
- HF_ASSETS_CACHE: /root/.cache/huggingface/assets
- HF_TOKEN_PATH: /root/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
Wauplin commented 1 month ago

Hi @XiaofuHuang, thanks for reporting this issue. It looks like this can happen in some rare cases because of a concurrency issue. I opened https://github.com/huggingface/huggingface_hub/pull/2288 to fix this. Since .gitignore file creation is not too important, I think it's best not to raise an error at this stage if we can't create it.