luoyoucode
commented
2 years ago The problem is “multiprocessing” module， "num_workers" is modified as following:
train_loader = DataLoader( train_dataset, config['batch_size'], shuffle=True, num_workers=0, pin_memory=True)
Hello Thank you for your awesome work. When I'm trying to run train_classifier.py code with miniImageNet dataset, I'm receiving the followed error:
./save\classifier_mini-imagenet_resnet12 exists, remove? ([y]/n): n mini-imagenet ./materials\mini-imagenet train dataset: torch.Size([3, 80, 80]) (x38400), 64 mini-imagenet ./materials\mini-imagenet val dataset: torch.Size([3, 80, 80]) (x18748), 64 mini-imagenet ./materials\mini-imagenet fs dataset: torch.Size([3, 80, 80]) (x12000), 20 num params: 8.0M Traceback (most recent call last): File "train_classifier.py", line 281, in main(config) File "train_classifier.py", line 148, in main for data, label in tqdm(train_loader, desc='train', leave=False): File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\site-packages\tqdm\std.py", line 1171, in iter for obj in iterable: File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 352, in iter return self._get_iterator() File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator return _MultiProcessingDataLoaderIter(self) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 801, in init w.start() File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'MiniImageNet.init..convert_raw' Traceback (most recent call last): File "", line 1, in File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\nusra\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 115, in _main self = reduction.pickle.load(from_parent) EOFError: Ran out of input
The config as follows:
{'train_dataset': 'mini-imagenet', 'train_dataset_args': {'split': 'train', 'augment': 'resize'}, 'val_dataset': 'mini-imagenet', 'val_dataset_args': {'split': 'train_phase_val'}, 'fs_dataset': 'mini-imagenet', 'fs_dataset_args': {'split': 'test'}, 'eval_fs_epoch': 5, 'model': 'classifier', 'model_args': {'encoder': 'resnet12', 'encoder_args': {}, 'classifier': 'linear-classifier', 'classifier_args': {'n_classes': 64}}, 'batch_size': 8, 'max_epoch': 100, 'optimizer': 'sgd', 'optimizer_args': {'lr': 0.1, 'weight_decay': 0.0005, 'milestones': [90]}, 'save_epoch': 5, 'visualize_datasets': True}
I would appreciate it if you give me any suggestions. Thank you