VPC CI of lower percentile missing

ggPMXdevelopment / ggPMX

ggPMX R package

37 stars 11 forks source link

VPC CI of lower percentile missing #298

Open tmss1 opened 1 year ago

tmss1 commented 1 year ago

Describe the bug CI for lower percentile missing in VPC

To Reproduce

ctr2 %>% 
  pmx_plot_vpc(
    scale_y_log10 = T,
    bin = pmx_vpc_bin(style="equal"),
  )

There are warning messages about NaN values:

Not sure if they are from the log-transformation of very small values, and if are causing the problem? Is it possible to check this and may exclude them for computation of CI?

mattfidler commented 1 year ago

My guess is many of the simulated values give NA or it is highly variable, which makes the lower value confidence interval not show or be reliable.

Depending on the model, it could be simulating values that are negative and then gives the bad values. If that is the case, you could try a log-normal distribution of residuals to see if this helps the VPC.

I'm unsure if this can be solved or not.

tmss1 commented 1 year ago

Thanks @mattfidler, As a follow up, the CIs are showing okay for a narrower prediction interval (e.g., 25th to 75th as attached), so it seems to support your suspicion that the simulated values are highly variable and some are too low or give NA values. The error model is only proportional so there should not be negative values.

tynsci commented 1 year ago

@tmss1

thanks for bringing up this issue
it has so far not been possible to reproduce the missing lower percetile Confidence Interval using available data
if there is any data you could share (here or privately) that would allow the behavior to be reproduced, then it may be possible to analyse it (otherwise we would try to generate some synthetic data)

tmss1 commented 1 year ago

Thanks @tynsci, Would you be able to start the investigation with some synthetic data first please as it may be difficult to share the data?