Reproducing diagnoses and procedures prediction results

bvanaken / clinical-outcome-prediction

Code for the EACL 2021 Paper: Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration

Apache License 2.0

88 stars 28 forks source link

Reproducing diagnoses and procedures prediction results #14

Open junayed-hasan opened 1 year ago

junayed-hasan commented 1 year ago

Hi there. Amazing codebase by the way!

I am trying to reproduce the results from your provided scripts and example configurations. However, I could easily reproduce the results for mortality prediction and length of stay in under 10 epochs. But for diagnoses and procedures, the AUROC is far from the results in the first ten epochs.

I believe this happens because of higher number of classes. But the rate of increase is very low. Did you achieve the reported results in under 200 epochs? Any insight on this would be very helpful and appreciated. Thanks!

ashishakkumar commented 3 months ago

@junayed-hasan Can you please share the script for the diagnoses and procedures and how you evaluated them. Thank you

bvanaken commented 2 months ago

Hi, Sorry, for only coming back to this now. It's been a while since I've worked on these topics. In general, for the diagnosis and procedure classification, the results were less stable regarding hyperparameters (which is also due to the high number of classes). So, I would definitely recommend an HP search. But in general, the number of epochs was rather high, I would say between 50-200. I did early stopping using a validation set, so the number varied.