search

mikevoets / jama16-retina-replication

JAMA 2016; 316(22) Replication Study
https://doi.org/10.1371/journal.pone.0217541
MIT License
110 stars 37 forks source link

Empty tf-records files #4

Closed agaldran closed 6 years ago

agaldran commented 6 years ago

Hi,

I'm sorry about asking this, but I have no knowledge about how tf-records work, and therefore have no clue on how to debug this.

When starting the training, the datasets appear to be empty. After checking the folder data/eyepacs/bin2/train, it seems that tf-records files are empty. Would you have a hint on what could have gone wrong here?

image

The same goes for the dimensions.txt file I was mentioning in a previous issue, and also for those 0 and 1 folders, so this may be the root of that problem I had also..

Thank you very much for your help. Cheers,

Adrian

mikevoets commented 6 years ago

Hi again! This seems to be the reason why your dimensions.txt file was empty.

Can you tell me if the directory data/eyepacs/pool contains 5 folders [0-4] with images inside?

agaldran commented 6 years ago

Hi, thanks for your quick reply, Mike. It does not, it only contains a test and a train folder, and they are empty:

image

Cheers,

Adrian

mikevoets commented 6 years ago

Wow, that's strange. Seems like python preprocess_eyepacs.py --data_dir="$pool_dir" on line 145 of eyepacs.sh did not run. Can you try to run that script with the help flag to verify that you don't get any error on running this Python script? $ python preprocess_eyepacs.py -h

agaldran commented 6 years ago

Ok, you are right, the python script call was failing silently. The reason was I was missing a dependency on matplotlib in my conda environment:

ModuleNotFoundError: No module named 'pylab'

After installing it, executing python preprocess_eyepacs.py -h appears to run smoothly. I am going to try now to re-run the entire sh script again.

Many thanks!

mikevoets commented 6 years ago

I added matplotlib to the list of dependencies and added a check in the preprocessing script for whether all dependencies are installed or not, so that this will not happen again. Thanks for opening this issue!

dragosbratu commented 5 years ago

For me, in windows, providing the full path to the command argument C:/path/to/the/file list fixed the problems(and empty files)