Closed cjacobs1 closed 5 years ago
I just reran the case and I identify this as an error in the preprocessing, just before saving the preprocessed data. Sadly it takes over an hour to get to the error. But at least I got a little bit more information that isn't truncated by the apply.py in the output trace above:
Traceback (most recent call last):
File "/mnt/synology/neuro/sil/bodyct-kaggle-grt123/main.py", line 251, in <module>
bug in 1.2.276.0.7230010.3.1.3.921054970.9700.1555410110.633
main(**config_submit)
File "/mnt/synology/neuro/sil/bodyct-kaggle-grt123/main.py", line 59, in main
use_existing=use_existing_preprocessing)
File "/mnt/synology/neuro/sil/bodyct-kaggle-grt123/preprocessing/full_prep.py", line 240, in full_prep
savenpy(i, filelist, prep_folder, data_path, use_existing=use_existing)
File "/mnt/synology/neuro/sil/bodyct-kaggle-grt123/preprocessing/full_prep.py", line 129, in savenpy
box = np.array([[np.min(xx), np.max(xx)], [np.min(yy), np.max(yy)],
File "/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2420, in amin
out=out, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py", line 29, in _amin
return umr_minimum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity
The responsible lines in the savenpy method:
newshape = np.round(np.array(Mask.shape) * spacing / resolution)
xx, yy, zz = np.where(Mask)
box = np.array([[np.min(xx), np.max(xx)], [np.min(yy), np.max(yy)],
[np.min(zz), np.max(zz)]])
My guess is that an invalid bounding box (probably empty) is extracted at that location, which will yield an error. I am currently retrying with the debugger at the point of error to investigate what the preprocessing mask looks like.
Ok, I just figured it out, the case is completely filled with a single value: -32768. The lung mask then becomes an empty image. Then the np.where(Mask) returns empty lists. Then it crashes on np.min
.
We might need some exception handling for this kind of case.
OK, thanks! I wonder what's going on. Perhaps an error in the dicom reading because it should not just contain -32768 values.. What dicom reader is used here? the diag dicom loader?
It uses the diag dataloader. It seems to happen both there and in Mevislab using the directdicomimport module.
OK, then there is something wrong with the dicom data. I will check tomorrow
@silvandeleemput I made a mistake when creating the input folders (the dicom loader was loading asynchronously, so it start writing out the new dicom before the pixeldata was actually loaded). I am recreating the data now.
The log shows this. I've put the test data in