about loo brier score for interval models?

[maurosc3ner]

Hi Eren,

I wonder if you have an example of evaluating the time-dependent brier score for a model like:

Surv(tstart,tstop,status)~covariates I have tested your paper+tutorial without success on this kind of model.

Thanks, Esteban

[ermeel86]

Hi Esteban,

intuitively I would think it works. To get concrete, we’d need to check two things:

a) is the time-dependent brier score applicable for interval censored data. b) if so, what is needed from the posterior in order to calculate it.

I admit I need to dive into this, as it’s been a while I’ve written this. Happy to figure this out together.

best, Eren

[maurosc3ner]

Hi Eren,

Thanks for answering. Let me add some more details. In brief, I followed your two tutorial-like papers about stan_surv. And using my time-dependent dataset (recurrent ED visits), I have been able to adapt and reproduce your code almost entirely including the RMST and the elpd (loo), but I cannot succeed in the brier score part. Mainly because a dependency, pec package is not able to handle intervals such as Surv(tstart,tstop,status)~covariates: Error in Hist(tstart, tstop, edvisit) : could not find function "Hist" Probably a manual estimation (in helpers.R)of brier or cindex could make the trick but I am not sure. I am also checking this approach for LTRC.

Although I am not an expert on bayes, in fact this is my first paper forcing myself to bayes, I am open to collaborate more.

Best, Esteban

[ermeel86]

I just checked the paper by Gerds et. al, and they explicitly assume right censoring. I do not know why and where in the proofs of the method this is needed. So I wouldn’t simply do an adhoc calculation, before checking the validity of the approach. Since I haven’t worked much with interval-censored data, I’m probably not a quick help here. I suggest to read the 2006 paper, and more recent papers citing, to check what has been done. Happy to discuss further, if I am of help. Best wishes, Eren

Am 10.10.2022 um 18:20 schrieb Esteban @.***>:



Hi Eren,

Thanks for answering. Let me add some more details. In brief, I followed your two tutorial-like papers about stan_surv. And using my time-dependent dataset (recurrent ED visits), I have been able to adapt and reproduce your code almost entirely including the RMST and the elpd (loo), but I cannot succeed in the brier score part. Mainly because a dependency, pec package is not able to handle intervals such as Surv(tstart,tstop,status)~covariates: Error in Hist(tstart, tstop, edvisit) : could not find function "Hist" Probably a manual estimation (in helpers.R)of brier or cindex could make the trick but I am not sure. I am also checking this approach for LTRC https://phcanalytics.github.io/coxnet-ltrc/example.html.

Although I am not an expert on bayes, in fact this is my first paper forcing myself to bayes, I am open to collaborate more.

Best, Esteban

— Reply to this email directly, view it on GitHub https://github.com/ermeel86/loobrierscore/issues/1#issuecomment-1273551635, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIDGJ2BGUP6IAVGUUVS7JDWCQ65PANCNFSM6AAAAAAQ2D7H7A. You are receiving this because you commented.