Lightning-AI / litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
https://lightning.ai
Apache License 2.0
10.47k stars 1.04k forks source link

FileNotFoundError with vicuna-7b-v1.5 #537

Closed ian00000 closed 1 year ago

ian00000 commented 1 year ago

I tried to download vicuna-7b-v1.5's weight, but it return FileNotFoundError.

I ran: !python scripts/download.py --repo_id lmsys/vicuna-7b-v1.5

The cell return: Fetching 6 files: 0% 0/6 [00:01<?, ?it/s] Traceback (most recent call last): File "/content/lit-gpt/scripts/download.py", line 74, in CLI(download_from_hub) File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 96, in CLI return _run_component(components, cfg_init) File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 181, in _run_component return component(cfg) File "/content/lit-gpt/scripts/download.py", line 45, in download_from_hub snapshot_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/_snapshot_download.py", line 239, in snapshot_download thread_map( File "/usr/local/lib/python3.10/dist-packages/tqdm/contrib/concurrent.py", line 69, in thread_map return _executor_map(ThreadPoolExecutor, fn, iterables, tqdm_kwargs) File "/usr/local/lib/python3.10/dist-packages/tqdm/contrib/concurrent.py", line 51, in _executor_map return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), kwargs)) File "/usr/local/lib/python3.10/dist-packages/tqdm/std.py", line 1182, in iter for obj in iterable: File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator yield _result_or_cancel(fs.pop()) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel return fut.result(timeout) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in get_result raise self._exception File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/_snapshot_download.py", line 214, in _inner_hf_hub_download return hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1427, in hf_hub_download _check_disk_space(expected_size, local_dir) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 975, in _check_disk_space target_dir_free = shutil.disk_usage(target_dir).free File "/usr/lib/python3.10/shutil.py", line 1331, in disk_usage st = os.statvfs(path) FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/lmsys/vicuna-7b-v1.5-16k'

carmocca commented 1 year ago

This looks like a bug in https://github.com/huggingface/huggingface_hub. I'm able to run the command using huggingface-hub==0.16.4. What's your version?

Do you have a checkpoints/ directory already?

ian00000 commented 1 year ago

It worked fine with huggingface-hub==0.16.4. Thanks a lot

carmocca commented 1 year ago

For curiosity, which version did you have? Maybe we need to pin it