search

MadryLab / robustness

A library for experimenting with, training and evaluating neural networks, with a focus on adversarial robustness.
MIT License
903 stars 181 forks source link

RuntimeError: Found 0 files in subfolders of: ./data/imagenet2012/images/val Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif #101

Closed g-griffin closed 3 years ago

g-griffin commented 3 years ago

After downloading the images from ImageNet and following the folder structure specified here (https://robustness.readthedocs.io/en/latest/example_usage/custom_imagenet.html) I get this error:

==> Preparing dataset custom_imagenet..
Traceback (most recent call last):
  File "C:\Users\G\anaconda3\envs\my_neuralnet\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-9c02721a0ee5>", line 1, in <module>
    runfile('C:/Users/G/PycharmProjects/my_neuralnet/src/my_mini_imagnet.py', wdir='C:/Users/Gretchen/PycharmProjects/my_neuralnet/src')
  File "C:\Program Files\JetBrains\PyCharm 2021.1.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2021.1.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/G/PycharmProjects/my_neuralnet/src/my_mini_imagnet.py", line 14, in <module>
    train_loader, test_loader = custom_dataset.make_loaders(workers=num_workers, batch_size=batch_size)
  File "C:\Users\G\anaconda3\envs\my_neuralnet\lib\site-packages\robustness\datasets.py", line 160, in make_loaders
    return loaders.make_loaders(workers=workers,
  File "C:\Users\G\anaconda3\envs\my_neuralnet\lib\site-packages\robustness\loaders.py", line 59, in make_loaders
    test_set = folder.ImageFolder(root=test_path, transform=transform_test,
  File "C:\Users\G\anaconda3\envs\my_neuralnet\lib\site-packages\robustness\tools\folder.py", line 213, in __init__
    super(ImageFolder, self).__init__(root, loader, IMG_EXTENSIONS,
  File "C:\Users\G\anaconda3\envs\my_neuralnet\lib\site-packages\robustness\tools\folder.py", line 92, in __init__
    raise(RuntimeError("Found 0 files in subfolders of: " + root + "\n"
RuntimeError: Found 0 files in subfolders of: ./data/imagenet2012/images/val
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif

val contains the 50000 validation images from ImageNet and train contains the original structure from ImageNet. All of the files from ImageNet are .JPEG.

src -> data -> imagenet2012 -> images -> train src -> data -> imagenet2012 -> images -> val

Windows 10 Pro Pycharm 2021.2 Pro Python 3.8.10 robustness 1.2.1post2

in_path = './data/imagenet2012/images/'
in_info_path = './data/imagenet2012/info/'

in_hier = ImageNetHierarchy(in_path, in_info_path)
superclass_wnid = common_superclass_wnid('mixed_10')
class_ranges, label_map = in_hier.get_subclasses(superclass_wnid, balanced=True)

custom_dataset = datasets.CustomImageNet(in_path, class_ranges)
train_loader, test_loader = custom_dataset.make_loaders(workers=num_workers, batch_size=batch_size)
g-griffin commented 3 years ago

Found the cause and solution here: (https://github.com/MadryLab/robustness/issues/95)

The val images need to be preprocessed into folders like the train images.