RobustBench / robustbench

RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
https://robustbench.github.io
Other
669 stars 99 forks source link

Error when loading custom ImageNet dataset. #124

Closed byerose closed 1 year ago

byerose commented 1 year ago
x_test, y_test = load_imagenet(n_examples=50,data_dir='/work/imagenet_robust/')

It report error:

KeyError                                  Traceback (most recent call last)
Cell In[36], line 1
----> 1 x_test, y_test = load_imagenet(n_examples=50,data_dir='/work/imagenet_robust/')
      3 model = load_model(model_name='Salman2020Do_R18', dataset='imagenet', threat_model='Linf')

File ~/anaconda3/envs/nn/lib/python3.9/site-packages/robustbench/data.py:135, in load_imagenet(n_examples, data_dir, transforms_test)
    131 if n_examples > 5000:
    132     raise ValueError(
    133         'The evaluation is currently possible on at most 5000 points-')
--> 135 imagenet = CustomImageFolder(data_dir + '/val', transforms_test)
    137 test_loader = data.DataLoader(imagenet,
    138                               batch_size=n_examples,
    139                               shuffle=False,
    140                               num_workers=4)
    142 x_test, y_test, paths = next(iter(test_loader))

File ~/anaconda3/envs/nn/lib/python3.9/site-packages/robustbench/loaders.py:183, in CustomImageFolder.__init__(self, root, transform, target_transform, loader, is_valid_file)
    177 def __init__(self,
    178              root,
    179              transform=None,
    180              target_transform=None,
    181              loader=default_loader,
    182              is_valid_file=None):
--> 183     super(CustomImageFolder,
...
---> 23                         c.split('\n')[0]), class_to_idx[c.split('/')[0]])
     24           for c in fnames]
     26 return images

KeyError: 'n09468604'

directory of custom dataset

/work/imagenet_robust/
├── train
└── val

based on robustbench v1.1

byerose commented 1 year ago

I found it. The shell wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash did not work.