search

NYUMedML / CNN_design_for_AD

Code for "Generalizable deep learning model for early Alzheimer’s disease detection from structural MRIs"
GNU Affero General Public License v3.0
138 stars 37 forks source link

Reproducibility #4

Closed Grigor355 closed 1 year ago

Grigor355 commented 1 year ago

Hi, thanks for the great project. I am trying to reproduce the results. I have downloaded ADNI, and used .sh files to preprocess the dataset following ReadMe instructions. The model trained by config.yaml of the repo, I got 60% accuracy compared to 68% reported in the model evaluation notebook. Have you used another config.yaml file?

shengliu66 commented 1 year ago

Hi, Thank you for your attention to our paper, are you using the same train/ Val/ split and the same subjects as ours (shown in the datasets/files folder)?

Grigor355 commented 1 year ago

Yes, I used the run_convert.sh and respective preprocess.sh files for data preparation. Each split may miss up to 3 scans as sometimes clinica got stuck on them, but the difference is really small and it should not produce the difference of ~8%.

Grigor355 commented 1 year ago

The evaluation notebook was using the checkpoint with the lowest loss but best_micro is the one producing the highest score 68%.