search

OHDSI / PatientLevelPrediction

An R package for performing patient level prediction in an observational database in the OMOP Common Data Model.
https://ohdsi.github.io/PatientLevelPrediction
181 stars 88 forks source link

Adding functions that enable plotting the AUC and Calibration of multiple databases in one plot #458

Open asaelmans opened 1 month ago

asaelmans commented 1 month ago

Is your feature request related to a problem? Please describe. When performing external validation of an existing model on multiple databases, I ran into a problem when wanting to plot several databases' calibration in one calibration plot.

Describe the solution you'd like I would like two new functions: multiPlotCalibration and multiPlotROC. These functions should enable the plotting of the results of several databases into one.

Describe alternatives you've considered Diving into the code of the PLP package and extracting the datapoints used for the plots by launching the individual database's results files.

Additional context I added two examples of what these plots could look like in the future.

image

image

rossdwilliams commented 1 month ago

Hi Alex,

this is good. I think we want to be able to pass multiple objects of plpResult into the function and then to produce these plots.

It would also be good to have a facet option per model or per database so that we have the flexibility to provide both options (the plots can get quite busy).

can you make a branch off of develop and work on this there?

egillax commented 1 month ago

Thanks Alex for the issue.

How did you create the plots for your poster? Were you able to use the current plotting functions in the package or did you have to make one from scratch?

I think ideally you'd just have one flexible multiPlot function and it would use the underlying PLP functions for whatever it's plotting (supporting faceting). At least internally, then you could add multiPlotROC or whatever around that for the user. But if the current plotting functions are not flexible enough that's something we'd have to look at.

I also think for completeness in the future this should support at least:

I'd also like net benefit but I see we don't have a plotting function for that, it's only in the shiny.

But feel free to just implement whatever suits you for your needs now and we can make follow up issues for the rest. We just need to make sure it would be compatible with extending it in the future.