Error in "enumerate" DataLoader during testing

UtkarshBhardwaj007 commented 3 years ago

I've been trying to run the trained example using the following query -

python test.py --phase test --serial_test --name robotcar_2day --dataroot ./datasets/test_set --n_domains 2 --which_epoch 150 --loadSize 512

I am facing the following error in the same. It seems like in line 24, enumerate isn't returning anything. Following is the entire execution result.

------------ Options ------------- aspect_ratio: 1.0 autoencode: False batchSize: 1 checkpoints_dir: ./checkpoints dataroot: ./datasets/test_set display_id: 0 display_port: 8097 display_single_pane_ncols: 0 display_winsize: 256 fineSize: 256 gpu_ids: [0] how_many: 50 input_nc: 3 isTrain: False loadSize: 512 max_dataset_size: inf nThreads: 2 n_domains: 2 name: robotcar_2day ndf: 64 netD_n_layers: 4 netG_n_blocks: 9 netG_n_shared: 0 ngf: 64 no_flip: False norm: instance output_nc: 3 phase: test reconstruct: False resize_or_crop: resize_and_crop results_dir: ./results/ serial_test: True show_matrix: False use_dropout: False which_epoch: 150 -------------- End ---------------- ---------- Networks initialized ------------- ResnetGenEncoder( (model): Sequential( (0): ReflectionPad2d((3, 3, 3, 3)) (1): Conv2d(3, 64, kernel_size=(7, 7), stride=(1, 1)) (2): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) (5): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (6): PReLU(num_parameters=1) (7): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) (8): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (9): PReLU(num_parameters=1) (10): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (11): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (12): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (13): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) ) ) ResnetGenDecoder( (model): Sequential( (0): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (1): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (2): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (3): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (4): ResnetBlock( (conv_block): SequentialContext( (0): ReflectionPad2d((1, 1, 1, 1)) (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (3): PReLU(num_parameters=1) (4): ReflectionPad2d((1, 1, 1, 1)) (5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (6): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) ) ) (5): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1)) (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (7): PReLU(num_parameters=1) (8): ConvTranspose2d(128, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1)) (9): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) (10): PReLU(num_parameters=1) (11): ReflectionPad2d((3, 3, 3, 3)) (12): Conv2d(64, 3, kernel_size=(7, 7), stride=(1, 1)) (13): Tanh() ) ) Created 2 Encoder-Decoder pairs Number of parameters per Encoder: 5099143 Number of parameters per Deocder: 6565770

Traceback (most recent call last): File "", line 1, in File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="mp_main__") File "D:\anaconda3\envs\torchenv\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\anaconda3\envs\torchenv\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\anaconda3\envs\torchenv\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\ub226\Desktop\2day\ToDayGAN\test.py", line 24, in for i, data in enumerate(dataset): File "C:\Users\ub226\Desktop\2day\ToDayGAN\data\data_loader.py", line 21, in iter for i, data in enumerate(self.dataloader): File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 352, in iter__ return self._get_iterator() File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator return _MultiProcessingDataLoaderIter(self) File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 801, in init w.start() File "D:\anaconda3\envs\torchenv\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init prep_data = spawn.get_preparation_data(process_obj._name) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "D:\anaconda3\envs\torchenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Traceback (most recent call last): File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 872, in _try_get_data data = self._data_queue.get(timeout=timeout) File "D:\anaconda3\envs\torchenv\lib\multiprocessing\queues.py", line 105, in get raise Empty queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test.py", line 24, in for i, data in enumerate(dataset): File "C:\Users\ub226\Desktop\2day\ToDayGAN\data\data_loader.py", line 21, in iter for i, data in enumerate(self.dataloader): File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 435, in next data = self._next_data() File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 1068, in _next_data idx, data = self._get_data() File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in _get_data success, data = self._try_get_data() File "D:\anaconda3\envs\torchenv\lib\site-packages\torch\utils\data\dataloader.py", line 885, in _try_get_data raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e RuntimeError: DataLoader worker (pid(s) 17648) exited unexpectedly

AAnoosheh commented 3 years ago

Hi, I’m not 100% sure but it seems like either a problem reading the data, or something with new versions of pytorch.

I would make sure your data folder structure is as described in the README, and if that doesn’t help, maybe try an older pytorch version?

Thanks

UtkarshBhardwaj007 commented 3 years ago

The directory structure is just as is mentioned in the README. However, I an uncertain as to which versions of all the required libraries should I use. Could you maybe freeze your environment using pip or anaconda to generate a requirements.txt file?

AAnoosheh commented 3 years ago

Yeah I know I should have done that. I did this project while I was a master's student, still uneducated in the ways of proper engineering etiquette such as requirements.

I no longer have the environment I used for this project, as it was on a school computer server which is long gone.

But I'm pretty sure you can just use the latest Pytorch version from the date I initially uploaded this repo.

UtkarshBhardwaj007 commented 3 years ago

We were all in that phase once weren't we. Thanks for your replies and I will try to figure this out.

AAnoosheh / ToDayGAN

Error in "enumerate" DataLoader during testing #23