search

Gorilla-Lab-SCUT / TTAC

[NeurIPS 2022] Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering
https://arxiv.org/abs/2206.02721
MIT License
44 stars 4 forks source link

ModuleNotFoundError: No module named 'rich' #2

Closed Bostoncake closed 1 year ago

Bostoncake commented 1 year ago

I installed your gorilla-core module and it worked well. However, when constructing the ImageNet dataset for training, when dealing with the frost corruption, I received following exception:

[ WARN:0@46013.964] imread_('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost3.png'): can't open/read file: check file path/integrity [ WARN:0@46013.973] imread('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity[ WARN:0@46013.975] imread('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/frost/frost2.png'): can't open/read file: check file path/integrity Traceback (most recent call last): File "/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 198, in track from rich.progress import track ModuleNotFoundError: No module named 'rich'

I wonder how I could get the module rich, thanks! By the way, when constructing the corruption datasets, I also received warnings as:

libpng warning: iCCP: profile 'icc': 'wtpt': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'bkpt': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'rXYZ': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'gXYZ': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'bXYZ': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'dmnd': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'dmdd': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'vued': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'view': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'lumi': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'meas': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'tech': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'rTRC': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'gTRC': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'bTRC': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 0h: PCS illuminant is not D50

I wonder if it's normal? Thanks!

Bostoncake commented 1 year ago

image The warnings are everywhere, though.

Dyb3438 commented 1 year ago

I installed your gorilla-core module and it worked well. However, when constructing the ImageNet dataset for training, when dealing with the frost corruption, I received following exception:

[ WARN:0@46013.964] imread_('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost3.png'): can't open/read file: check file path/integrity [ WARN:0@46013.973] imread('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity[ WARN:0@46013.975] imread('/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/frost/frost2.png'): can't open/read file: check file path/integrity Traceback (most recent call last): File "/home/username/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 198, in track from rich.progress import track ModuleNotFoundError: No module named 'rich'

I wonder how I could get the module rich, thanks! By the way, when constructing the corruption datasets, I also received warnings as:

libpng warning: iCCP: profile 'icc': 'wtpt': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'bkpt': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'rXYZ': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'gXYZ': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'bXYZ': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'dmnd': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'dmdd': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'vued': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'view': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'lumi': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'meas': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'tech': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'rTRC': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 'gTRC': ICC profile tag start not a multiple of 4libpng warning: iCCP: profile 'icc': 'bTRC': ICC profile tag start not a multiple of 4 libpng warning: iCCP: profile 'icc': 0h: PCS illuminant is not D50

I wonder if it's normal? Thanks!

rich is a Python library for rich text and beautiful formatting in the terminal. You can install this module by pip install rich. Which type of corrupted images were generating while these warning messages were alerted?

Bostoncake commented 1 year ago

I'm sure that those messages appear when generating motion_blur images. I'm not sure about other corrupted images and I will try again.

Bostoncake commented 1 year ago

I installed rich and received these exceptions instead:

frost
Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--[ WARN:0@1.382] imread_('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost3.png'): can't open/read file: check file path/integrity [ WARN:0@1.385] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:-- [ WARN:0@1.390] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost4.jpg'): can't open/read file: check file path/integrity [ ] 0/196, elapsed: 0s, ETA:[ WARN:0@1.398] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost2.png'): can't open/read file: check file path/integrity [ WARN:0@1.398] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity [ WARN:0@1.456] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/frost/frost2.png'): can't open/read file: check file path/integrity Traceback (most recent call last):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 199, in track for task in track(tasks):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 169, in track sequence, total=total, description=description, update_period=update_period File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 1215, in track for value in sequence: File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise()
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem sample = self.transform(sample)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call img = t(img) File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224) AttributeError: 'NoneType' object has no attribute 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "utils/create_corruption_dataset.py", line 54, in for batch in gorilla.track(target_dataloader):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 203, in track for task in tasks:
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data()
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise()
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 199, in track for task in track(tasks):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 169, in track sequence, total=total, description=description, update_period=update_period
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 1215, in track for value in sequence:
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data()
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise()
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem
sample = self.transform(sample)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call
img = t(img)
File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224)
AttributeError: 'NoneType' object has no attribute 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem sample = self.transform(sample)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call img = t(img)
File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity)
File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224)
AttributeError: 'NoneType' object has no attribute 'shape'

Dyb3438 commented 1 year ago

I'm sure that those messages appear when generating motion_blur images. I'm not sure about other corrupted images and I will try again.

I found that, the similar situation also happens in my environment. You don't have to worry about this libpng warning, it doesn't affect our experiments.

Dyb3438 commented 1 year ago

I installed rich and received these exceptions instead:

frost Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--[ WARN:0@1.382] imread_('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost3.png'): can't open/read file: check file path/integrity [ WARN:0@1.385] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:-- [ WARN:0@1.390] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost4.jpg'): can't open/read file: check file path/integrity [ ] 0/196, elapsed: 0s, ETA:[ WARN:0@1.398] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost2.png'): can't open/read file: check file path/integrity [ WARN:0@1.398] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenetc/frost/frost1.png'): can't open/read file: check file path/integrity [ WARN:0@1.456] imread('/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/frost/frost2.png'): can't open/read file: check file path/integrity Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 199, in track for task in track(tasks): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 169, in track sequence, total=total, description=description, update_period=update_period File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 1215, in track for value in sequence: File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem sample = self.transform(sample) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call img = t(img) File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224) AttributeError: 'NoneType' object has no attribute 'shape' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "utils/create_corruption_dataset.py", line 54, in for batch in gorilla.track(target_dataloader): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 203, in track for task in tasks: File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/gorilla/utils/processbar.py", line 199, in track for task in track(tasks): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 169, in track sequence, total=total, description=description, update_period=update_period File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/rich/progress.py", line 1215, in track for value in sequence: File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 681, in next data = self._next_data() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data return self._process_data(data) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data data.reraise() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/_utils.py", line 461, in reraise raise exception AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem sample = self.transform(sample) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call img = t(img) File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224) AttributeError: 'NoneType' object has no attribute 'shape' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 232, in getitem sample = self.transform(sample) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 94, in call img = t(img) File "utils/create_corruption_dataset.py", line 27, in call x_corrupted = corruption_dict[self.corruption_name](x, self.severity) File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 259, in frost x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224) AttributeError: 'NoneType' object has no attribute 'shape'

This situation is more like you are missing the frost folder in imagenet_c package. The frost folder looks like this: https://github.com/hendrycks/robustness/tree/master/ImageNet-C/imagenet_c/imagenet_c/frost. This issue has been pointed out at: https://github.com/hendrycks/robustness/issues/4. You can solve this following below:

  1. Uninstall the previous imagenet_c package by pip uninstall imagenet_c
  2. Download the code directly at this repo
  3. Then run the installing command pip install -e . in the imagenet_c folder. (Refers to: https://github.com/hendrycks/robustness/issues/4#issuecomment-427226016) Thank you.
Dyb3438 commented 1 year ago

Please run pip install -e . under this folder: https://github.com/hendrycks/robustness/tree/master/ImageNet-C/imagenet_c.

Bostoncake commented 1 year ago

Please run pip install -e . under this folder: https://github.com/hendrycks/robustness/tree/master/ImageNet-C/imagenet_c.

Everything was solved and, thank you so much for the instructions! I could reproduce your results on ImageNet-C in my server now. There is still a question for me. I really want to know where did you find the corrupted versions of CIFAR datasets? I just want to know whether other papers also referred to the source on zenodo.org. Thank you and sorry for the bothering!

Bostoncake commented 1 year ago

By the way, I tried to reproduce the results on another server that I have access to. But I received these results:

(torch1121) xiongyizhe@ubuntu-KI4224G:~/TTT/TTAC/imagenet$ python utils/create_corruption_dataset.py Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 154, in libraries = load_library() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 143, in load_library raise IOError('cannot find library; tried paths: ' + repr(tried_paths)) OSError: cannot find library; tried paths: []

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "utils/create_corruption_dataset.py", line 1, in from imagenet_c import File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/init.py", line 3, in from .corruptions import File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 11, in from wand.image import Image as WandImage File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/image.py", line 18, in from . import assertions File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/assertions.py", line 155, in from .color import Color # noqa: E402 File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/color.py", line 10, in from .api import library File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 180, in 'Try to install:\n ' + msg) ImportError: MagickWand shared library not found. You probably had not installed ImageMagick library. Try to install: apt-get install libmagickwand-dev

I have no root access to this server. I wonder if you have encountered similar problems?

Dyb3438 commented 1 year ago

By the way, I tried to reproduce the results on another server that I have access to. But I received these results:

(torch1121) xiongyizhe@ubuntu-KI4224G:~/TTT/TTAC/imagenet$ python utils/create_corruption_dataset.py Traceback (most recent call last): File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 154, in libraries = load_library() File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 143, in load_library raise IOError('cannot find library; tried paths: ' + repr(tried_paths)) OSError: cannot find library; tried paths: [] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "utils/create_corruption_dataset.py", line 1, in from imagenet_c import File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/init.py", line 3, in from .corruptions import File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/imagenet_c/corruptions.py", line 11, in from wand.image import Image as WandImage File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/image.py", line 18, in from . import assertions File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/assertions.py", line 155, in from .color import Color # noqa: E402 File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/color.py", line 10, in from .api import library File "/home/xiongyizhe/miniconda3/envs/torch1121/lib/python3.7/site-packages/wand/api.py", line 180, in 'Try to install:\n ' + msg) ImportError: MagickWand shared library not found. You probably had not installed ImageMagick library. Try to install: apt-get install libmagickwand-dev

I have no root access to this server. I wonder if you have encountered similar problems?

libmagickwand-dev is necessary to install. Maybe you need to contact your server administrator to help you install?

Dyb3438 commented 1 year ago

Please run pip install -e . under this folder: https://github.com/hendrycks/robustness/tree/master/ImageNet-C/imagenet_c.

Everything was solved and, thank you so much for the instructions! I could reproduce your results on ImageNet-C in my server now. There is still a question for me. I really want to know where did you find the corrupted versions of CIFAR datasets? I just want to know whether other papers also referred to the source on zenodo.org. Thank you and sorry for the bothering!

You could download the CIFAR10-C / CIFAR100-C datasets following this repo, since my experience is built upon it.

export DATADIR=./data
sudo mkdir -p ${DATADIR} && cd ${DATADIR}
wget -O CIFAR-10-C.tar https://zenodo.org/record/2535967/files/CIFAR-10-C.tar?download=1
tar -xvf CIFAR-10-C.tar
wget -O CIFAR-100-C.tar https://zenodo.org/record/3555552/files/CIFAR-100-C.tar?download=1
tar -xvf CIFAR-100-C.tar
Bostoncake commented 1 year ago

Thanks for the reply! I think I'm all set for now.