baidu-research / NCRF

Cancer metastasis detection with neural conditional random field (NCRF)
Apache License 2.0
757 stars 184 forks source link

Pickle issue with prob_maps while generating heat maps #35

Closed OpenCv30 closed 5 years ago

OpenCv30 commented 5 years ago

HI, I am running pobs_map to generate the heatmap for 130 test images to reproduce the FROC. I started with normal configuration --GPU 0,1 and --num_workers 5 I am seeing the following error on Windows.

Traceback (most recent call last): for (data, x_mask, y_mask) in dataloader: File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 501, in iter return _DataLoaderIter(self) File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 289, in init w.start() File "C:\ProgramData\Miniconda3\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\ProgramData\Miniconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in init reduction.dump(process_obj, to_child) File "C:\ProgramData\Miniconda3\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) ValueError: ctypes objects containing pointers cannot be pickled Traceback (most recent call last): File "", line 1, in File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 115, in _main self = reduction.pickle.load(from_parent) EOFError: Ran out of input

But when I have changed the setting to - --GPU 0 and --num_workers 0 , It started running but very slow for one heatmap @level6 - it is showing 21hrs.

Can you help me on this?

yil8 commented 5 years ago

@OpenCv30 Sorry, but I've never tried using Windows. Can you try on using Linux machine?