search

jaeho3690 / LIDC-IDRI-Preprocessing

This is the preprocessing step of the LIDC-IDRI dataset
160 stars 43 forks source link

FileNotFoundError for "Clean" data #6

Closed harsharaman closed 3 years ago

harsharaman commented 3 years ago

Hi Jaeho, my name is Harsha. Thank you for this repository. I am using it to preprocess data for my segmentation model. My issue is that as soon as your script encounters the first "clean" data files, it throws the following error: Patient ID: LIDC-IDRI-0028 Dicom Shape: (512, 512, 141) Number of Annotated Nodules: 0 Clean Dataset LIDC-IDRI-0028
Traceback (most recent call last):
File "prepare_dataset.py", line 173, in test.prepare_dataset()
File "prepare_dataset.py", line 157, in prepare_dataset
np.save(patient_clean_dir_mask / mask_name, lung_mask) File "<__array_function__ internals>", line 6, in save
File "/home/ramanha/env3/lib/python3.5/site-packages/numpy/lib/npyio.py", line 541, in save
fid = open(file, "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'data/Clean/Mask/LIDC-IDRI-0028/LIDC-IDRI-0028/0028_CM001_slice000.npy'

jaeho3690 commented 3 years ago

Hi, have you made the data/Clean directory?

harsharaman commented 3 years ago

data/Clean was present from before. Do I have to explicitly make it?

jaeho3690 commented 3 years ago

The error that is throwing is Filenotfounderror. 'data/Clean/Mask' this folder should exist in your directory as the clean .npy files are saved in that directory.

harsharaman commented 3 years ago

data/Clean/Image and data/Clean/Mask folders exist and I have checked them. This is why I got confused about this error. The folders are present.

jaeho3690 commented 3 years ago

It seems that the script is trying to make a same folder. "LIDC-IDRI-0028/LIDC-IDRI-0028/" -> there should only be one folder per patient. So it should actually be 'data/Clean/Mask/LIDC-IDRI-0028/0028_CM001_slice000.npy'

jaeho3690 commented 3 years ago

Please check the config settings and codes that are making directories

harsharaman commented 3 years ago

Hi, so I have found the error: In lines 152 and 153 of prepare_dataset.py, there's an extra {} which creates the folder. nodule_name = "{}/{}_CN001_slice{}".format(pid,pid[-4:],prefix[slice]) mask_name = "{}/{}_CM001_slice{}".format(pid,pid[-4:],prefix[slice])

Please remove it and this issue is resolved :D

jaeho3690 commented 3 years ago

Great to hear that! I am not sure why such a problem occurs because it worked for mine but I will have a look at it! Thanks a lot :) and good luck!