Error - Meta file not found or corrupted

[Esaada]

Hey I’m trying to work with torchbench in order to benchmark on imagent. I'm getting this kind of error is:

RuntimeError: ('Meta file not found or corrupted.', 'You can use download=True to create it.')

(Full error is at the end of the post)

I dived into the source code and I couldn’t understand the why is the meta file, whatever it is, missing, It's important to say that I'm using this dataset and path for many other projects I'm working on, so it's weird that something is missing.

This is how I called torchbench:

ImageNet.benchmark(
model=model,
paper_model_name=‘FixResNeXt-101-32x48d’,
paper_arxiv_id=‘1906.06423’,
input_transform=input_transform,
data_root=“imagenet_dataset”,
batch_size=32,
num_gpu=1
)

Any idea?

Full error: Benchmarking on ImageNet... [15/1814] Downloading http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_devkit_t12.tar.gz to /dataset-imagenet-ilsvrc2012/ILSVRC2012_devkit_t12.tar.gz 0it [00:00, ?it/s]Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/torchbench/image_classification/imagenet.py", line 185, in benchmark download=True, File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/imagenet.py", line 55, in init
self.download()
File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/imagenet.py", line 75, in download
md5=archive_dict['md5']) File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/utils.py", line 248, in download_and_extract_archive download_url(url, download_root, filename, md5) File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/utils.py", line 96, in download_url
raise e
File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/utils.py", line 84, in download_url reporthook=gen_bar_updater() File "/usr/lib/python3.6/urllib/request.py", line 248, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp: File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.6/urllib/request.py", line 570, in error return self._call_chain(args) File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(args) File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred: Traceback (most recent call last): File "evaluation_script.py", line 87, in num_gpu=1 File "/usr/local/lib/python3.6/dist-packages/sotabenchapi/core/inputs.py", line 85, in regular_evaluation func(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/torchbench/image_classification/imagenet.py", line 193, in benchmark download=False, File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/imagenet.py", line 56, in init wnid_to_classes = self._load_meta_file()[0] File "/usr/local/lib/python3.6/dist-packages/torchvision/datasets/imagenet.py", line 108, in _load_meta_file "You can use download=True to create it.") RuntimeError: ('Meta file not found or corrupted.', 'You can use download=True to create it.')

[RJT1990]

Thanks for reporting the issue! It seems like you are running this locally - is this correct?

From your log, it appears the issue is:

• You can't find the URL of the ImageNet dataset -> "HTTP Error 404: Not Found"
• As a result -> no meta file found -> ''Meta file not found or corrupted.'

The torchbench library uses the torchvision ImageNet dataset. Could you try seeing if the error persists for torchvision when you load the ImageNet data directly through torchvision (rather than going through the benchmark interface)? Something like:

import torchvision.datasets as datasets
input_transform = ...
imagenet = datasets.ImageNet("imagenet_dataset", split="val", transform=input_transform, target_transform=None, download=True,)

This should isolate whether it's an issue with trying to get the dataset from torchvision from your machine.

[Esaada]

I'm connecting through ssh to my server, and run from there, I have the dataset locally. I already worked with this dataset using torchvision. why should I need a URL for Imagenet?

[RJT1990]

Hey @BarakBat. What I’m trying to verify is that you are getting the same error by using the torchvision code quoted above. Torchbench uses torchvision, and the problem is coming from Dataset initialisation. Could you try the torchvision code above and see if you can reproduce the error?

[Esaada]

I understand what you mean and you're correct it's a pytorch problem. Thansk.