search

MIC-DKFZ / medicaldetectiontoolkit

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.
Apache License 2.0
1.3k stars 297 forks source link

Memory Error when preprocessing LIDC file #32

Closed ivanwilliammd closed 5 years ago

ivanwilliammd commented 5 years ago

Hi Sir Paul, thank you for your marvelous work

I already convert LIDC dataset to nrrd (nifty) file, then when I want to preprocessed it using preprocessing.py for the first time, I got this error:

warn("Anti-aliasing will be enabled by default in skimage 0.15 to " /home/ivanwilliam/.virtualenvs/virtual-py3/lib/python3.5/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images. warn("Anti-aliasing will be enabled by default in skimage 0.15 to " multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar return list(map(args)) File "preprocessing.py", line 61, in pp_patient img_arr = resample_array(img_arr, img.GetSpacing(), cf.target_spacing) File "preprocessing.py", line 48, in resample_array img = src_imgs.astype(float) MemoryError """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "preprocessing.py", line 137, in p1 = pool.map(pp_patient, enumerate(paths), chunksize=1) File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get raise self._value MemoryError

Thanks in advance Sir.

Note: command I run is python preprocessing.py with change in configs.py directory settings

My computer spec: Intel i7-7700, 16GB RAM, NVIDIA GTX1050Ti 4GB and I already try to downgrade my NumPy to 1.14.5 based on https://github.com/MIC-DKFZ/batchgenerators README.md recommendation

ivanwilliammd commented 5 years ago

Update: I already downsize the process pool pool = Pool(processes=12) --> pool = Pool(processes=2), and it works with a little lag until it shows the new error:

processing 0005a (0.664062, 0.664062, 2.5) (133, 512, 512) /home/ivanwilliam/.virtualenvs/virtual-py3/lib/python3.5/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images. warn("Anti-aliasing will be enabled by default in skimage 0.15 to " processing 0004a (0.822266, 0.822266, 1.25) (241, 512, 512) /home/ivanwilliam/.virtualenvs/virtual-py3/lib/python3.5/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images. warn("Anti-aliasing will be enabled by default in skimage 0.15 to " processing 0009a (0.78125, 0.78125, 1.25) (256, 512, 512) suppressed roi! ['0005a_0_00000069_00000069.nii.gz'] processing 0003a (0.820312, 0.820312, 2.5) (140, 512, 512) suppressed roi! ['0009a_0_00000143_00000143.nii.gz'] suppressed roi! ['0009a_0_00000142_00000142.nii.gz'] processing 0008a (0.78125, 0.78125, 2.5) (133, 512, 512) processing 0007a (0.78125, 0.78125, 2.5) (145, 512, 512) suppressed roi! ['0007a_1_00000125_00000125.nii.gz'] processing 0010a (0.878906, 0.878906, 1.25) (277, 512, 512) suppressed roi! ['0003a_3_00000053_00000053.nii.gz'] processing 0001a (0.703125, 0.703125, 2.5) (133, 512, 512) multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar return list(map(args)) File "preprocessing.py", line 78, in pp_patient rater_labels = [df[df.NoduleID == int(ii)].Malignancy.values[0] for ii in nodule_ids] File "preprocessing.py", line 78, in rater_labels = [df[df.NoduleID == int(ii)].Malignancy.values[0] for ii in nodule_ids] ValueError: invalid literal for int() with base 10: '' """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "preprocessing.py", line 138, in p1 = pool.map(pp_patient, enumerate(paths), chunksize=1) File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get raise self._value ValueError: invalid literal for int() with base 10: ''

Does by changing process pool size will affect the converted data? Thank you in advance Sir.

ivanwilliammd commented 5 years ago

I already resolved this problem by manually remove the NoduleID 00000000 from characteristics.csv.