search

OHDSI / DeepPatientLevelPrediction

An R package for performing patient level prediction using deep learning in an observational database in the OMOP Common Data Model.
https://ohdsi.github.io/DeepPatientLevelPrediction
10 stars 4 forks source link

Add MLP model using the estimator #29

Closed egillax closed 1 year ago

egillax commented 2 years ago

Lately I've been trying to get the DeepNNTorch models to work. There were some problems with them. After working on it for a few days and getting it to train, when I test on real data I get AUCs of 0.5 . Which to me suggest there's something more wrong with it, which will take more time to chase down.

Since I already have a training loop encoded in the estimator I quickly added a MLP model using the estimator. That immediately does better with auc's above 0.8.

My suggestion is thus:

Remove the DeepNNTorch models and use the new MLP instead. I know at some point someone expressed that they wanted to keep them because they are simple. But that is moot if they don't work.

So if someone objects to me removing them please comment here, otherwise I'll replace them with the MLP for the first release.

@jreps @lhjohn @chandryou @ted9219

ChungsooKim commented 2 years ago

Hi, Egill Thank you for your testing. I developed the DeepNNTorch model quite a long time ago, so It would have a compatibility issue with the current PLP. I will try a test and will share it soon. Thanks.

egillax commented 2 years ago

Hi Kim,

I moved the DeepNNTorch model to a branch. I already did a lot of changes including:

If you want to test it you can test it in the branch. In the meantime I added the MLP using the estimator class. It's performing pretty well, slightly lower than the ResNet on the one prediction problem we used in the abstract.

egillax commented 1 year ago

Closing this since the MLP is included now and uses the estimator to fit. @ted9219 if you still want to look at the DeepNNTorch model you can open another issue for that.