search

TencentYoutuResearch / AnomalyDetection_Real-IAD

Apache License 2.0
47 stars 0 forks source link

Having trouble to download the data from huggingface #7

Open ghost opened 4 months ago

ghost commented 4 months ago

Hi !

I just tried to download Real-IAD.

But an error occurred constantly..

the error is..

  File "/home/jisu/anaconda3/lib/python3.9/site-packages/huggingface_hub/hf_file_system.py", line 638, in get_file
    http_get(
  File "/home/jisu/anaconda3/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 570, in http_get
    raise EnvironmentError(
OSError: Consistency check failed: file should be of size 23215841838 but has size 10001900996 ((…)f5878d/realiad_raw/rolled_strip_base.zip).
We are sorry for the inconvenience. Please retry with `force_download=True`.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

The code I used is 

from datasets import load_dataset
# Real-IAD/Real-IAD 데이터셋 로드 및 다운로드
dataset = load_dataset("Real-IAD/Real-IAD", cache_dir="/SSD2/Datasets/RealIAD")

I downloaded the data above amount.

Resolving data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 477.45it/s]
Downloading data:  77%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                                | 93/121 [1:51:55<5:05:29, 654.63s/files]
Downloading data:  87%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎                  | 105/121 [7:30:11<1:08:36, 257.25s/files]

Any solution??

thanks!