search

nliulab / AutoScore

AutoScore: An Interpretable Machine Learning-Based Automatic Clinical Score Generator
31 stars 4 forks source link

cross-validation? #8

Open shlid007 opened 1 year ago

shlid007 commented 1 year ago

Hi - is cross-validation possible instead of using a separate validation set (without having to modify the core delivered code)? Or what about bootstrapping?

fengx13 commented 1 year ago

Our tutorial has covered cross-validation part. You may refer to https://nliulab.github.io/AutoScore/04-autoscore.html#demo2

shlid007 commented 1 year ago

Thank you Dr. Feng! To do that on the final model, not just training, I believe Dr. Liu said to split my test dataset into x partitions and do CV that way, correct? Example: to do 5-fold CV, split test dataset into five portions and run the model against each portion (from my test set).

Reason for doing CV on a test set is, I want to collect multiple observations on the final model to do statistical testing of differences (across, say, logistic regression, random forest, and AutoScore).

Hopefully my explanation makes sense ... If I interpreted the approach incorrectly, please let me know.

Thanks, Sue Lhymn

From: XIE FENG @.> Sent: Tuesday, August 29, 2023 2:01 AM To: nliulab/AutoScore @.> Cc: Sue Lhymn @.>; Author @.> Subject: Re: [nliulab/AutoScore] cross-validation? (Issue #8)

Our tutorial has covered cross-validation part. You may refer to https://nliulab.github.io/AutoScore/04-autoscore.html#demo2

— Reply to this email directly, view it on GitHubhttps://github.com/nliulab/AutoScore/issues/8#issuecomment-1696819833, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A3MFT2NELDZRVP4UF3A7CU3XXWASPANCNFSM6AAAAAA2NUVNR4. You are receiving this because you authored the thread.Message ID: @.***>