YerevaNN / mimic3-benchmarks

Python suite to construct benchmark machine learning datasets from the MIMIC-III 💊 clinical database.
https://arxiv.org/abs/1703.07771
MIT License
805 stars 329 forks source link

replication of model prediction - deviation of ~2% random seeds missing #128

Open partizanos opened 2 years ago

partizanos commented 2 years ago

Hello,

I try to replicate the training and evaluation using LSTM network for the phenotype benchmark. I would like to ask if there are specific random seeds set in your system for the model training. (for the folders I saw the code provided them but for training no) ie from tensorflow import set_random_seed set_random_seed(2) I get very close results (+/- 0.02 deviation) and I suspect this might be the issue. But let me know if there was some change image screenshot refers to this version of the paper:https://arxiv.org/pdf/1703.07771.pdf

Some more potential reasons for non replicability

Thank you again for your work and active maintenance of this repository.