search

lfz / DSB2017

The solution of team 'grt123' in DSB2017
MIT License
1.23k stars 420 forks source link

All false Mask (= m1+m2) in preprocessing for some scans from stage2 #74

Closed max-ilse closed 6 years ago

max-ilse commented 6 years ago

Hello everybody,

Since 2 days, I am trying to run main.py to see if everything is working correctly. Unfortunately, I am running into problems during the preprocessing for some files, e.g. ac7493533e667b077b7180642fb1dc4d

The problem is that in line 70 of full_prep.py: im, m1, m2, spacing = step1_python(os.path.join(data_path,name))

All elements in m1 and m2 are False. Which results in empty arrays for xx,yy,zz in line 74: xx,yy,zz= np.where(Mask)

Afterwards the code fails since there is no np.min() for empty arrays.

First I thought maybe the file got corrupted during download so I downloaded stage2 again, but that didn't help.

I don't use the right versions for some of the python modules, e.g. numpy 1.11.13 doesn't work (mkl problems - still investigating) therefore I am using 1.12, I have hard times to believe that this is causing problems since the preprocessing is working for most of the files.

Has anybody an idea what else I could try? Thanks in advance!

lfz commented 6 years ago

Hi, it might be the threshold problem, i.e. the volume of the lung if smaller than the threshold. But I am not sure, I suggest you to debug the step_python1 function line by line, the ideas of the function is discribed in our paper, and we discribed the output of each module, you should see similar results in those normal cases.

2018-04-10 6:05 GMT+08:00 max-ilse notifications@github.com:

Hello everybody,

Since 2 days, I am trying to run main.py to see if everything is working correctly. Unfortunately, I am running into problems during the preprocessing for some files, e.g. ac7493533e667b077b7180642fb1dc4d

The problem is that in line 70 of full_prep.py: im, m1, m2, spacing = step1_python(os.path.join(data_path,name))

All elements in m1 and m2 are False. Which results in empty arrays for xx,yy,zz in line 74: xx,yy,zz= np.where(Mask)

Afterwards the code fails since there is no np.min() for empty arrays.

First I thought maybe the file got corrupted during download so I downloaded stage2 again, but that didn't help.

I don't use the right versions for some of the python modules, e.g. numpy 1.11.13 doesn't work (mkl problems - still investigating) therefore I am using 1.12, I have hard times to believe that this is causing problems since the preprocessing is working for most of the files.

Has anybody an idea what else I could try? Thanks in advance!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lfz/DSB2017/issues/74, or mute the thread https://github.com/notifications/unsubscribe-auth/AIigQ0qJVa3bIgwRZVii3ie1MSaM2-4Lks5tm9sXgaJpZM4TNTLT .

-- 廖方舟 清华大学医学院 Liao Fangzhou School of Medicine Tsinghua University Beijing 100084 China

Mango233 commented 6 years ago

Hello, I'd like to ask how to download stage2 data，Thanks

max-ilse commented 6 years ago

Thanks for replying! I used the kaggle api twice and did one direct download (from the kaggle page).

Right now I am running the preprocessing on the training set (stage1) and so far it seems fine - it did 200 examples without a problem. So maybe in the end my stage 2 data is corrupted.

Any other ideas?

On Tue, Apr 10, 2018, 3:52 PM Mango notifications@github.com wrote:

Hello, I'd like to ask how to download stage2 data，Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lfz/DSB2017/issues/74#issuecomment-380105941, or mute the thread https://github.com/notifications/unsubscribe-auth/AJt5ohCmjHN7UZynZxjEswYOoanCNj3iks5tnLkggaJpZM4TNTLT .

max-ilse commented 6 years ago

In the end it seems like my environment was causing the preprocessing to fail. Finally, I got it to work. Here is my conda environment:

name: nodules channels: