search

drizopoulos / JMbayes2

Extended Joint Models for Longitudinal and Survival Data
https://drizopoulos.github.io/JMbayes2/
79 stars 23 forks source link

Confidence interval for tvAUC #49

Closed ahulman closed 1 year ago

ahulman commented 1 year ago

Hi Dimitris, It would be super very to have the option to return not only the AUC by the tvAUC function, but also a 95% CI. Do you have any plans of adding this in the future? I am comparing the performance of two models, but it is quite difficult to judge the difference between the AUCs without knowing their precision. I guess I could do a bootstrap as the simplest solution, but wanted to hear your thoughts before trying that. Cheers, Adam

drizopoulos commented 1 year ago

Hi Adam,

This is not implemented yet, but in principle, it is not very difficult to do because we can extract the Monte Carlo sample of the estimated from the joint model cumulative risk probabilities.

However, note that this would not be suitable for comparing models. That is, if from model M1 you have an AUC1 with 95% CI (a, b), and from model M2 you have an AUC2 with 95% CI (c, d), you cannot simply see if AUC2 > AUC1 by seeing if the 95% CIs overlap or not. You will need the 95% CI for the difference AUC2 - AUC1. And to get this, you will need to use Bootstrap.

ahulman commented 1 year ago

Thanks for the response, it would be great to have that option in the package. Good point with model comparison.