Unexpected results for PredictionBand

[JorisDeRidder]

• UltraNest version: 3.5.7
• Python version: 3.9.16
• Operating System: OSX

Description

Given its name, I expected UltraNest's PredictionBand to give a band derived from the posterior predictive distribution, but this is not what it seems to compute. The following example illustrates this.

• The dots are a synthetic dataset created with

  rng = np.random.default_rng(14)
  xobs = np.linspace(0, 1, 50)
  yobs = 1 + 2 * xobs + rng.normal(0.0, 0.3, 50)

  and which was modeled with a simple straight line using UltraNest

• The lightblue shaded region is the 95% prediction band computed by UltraNest using the code

  band.shade(q=0.475, color='lightsteelblue', alpha=0.6)            # [0.5-0.475, 0.5+0.475] = [0.025, 0.975] 

• The green dot-dashed lines are the frequentist 95% confidence interval of the responses
• The purple dashed lines are the frequentist 95% prediction interval of the responses. They coincide with the 95% HDI of the posterior predictive distribution computed with PyMC or PyStan.

Am I correct in concluding that PredictionBand does not use the posterior predictive distribution, and if so, could this feature be added to UltraNest?

[JohannesBuchner]

I am not sure I understand your question. PredictionBand plots quantiles of whatever you feed it with .add()

It seems here you gave it the lines. From your question I guess you want to give it newly sampled data?

[JorisDeRidder]

Thanks for the very quick answer. That's indeed the cause of the discrepancy, which I should have seen but which I overlooked when copying the examples given in the documentation. You might want to consider changing the example about "Time series fitting" in UltraNest's documentation. The line

band = PredictionBand(t_grid)

suggests you want to construct a prediction band, which I suspect many users relate to the posterior predictive distribution. But in the end that's not what is computed in that example, so this could be confusing.

[JohannesBuchner]

As far as I understand, "posterior predictive distribution" is a concept not restricted to the data, indeed, https://en.wikipedia.org/wiki/Posterior_predictive_distribution says "posterior predictive distribution of x" (where x is the data) and suggests that other unobserved quantities also have a posterior predictive distribution.

[JohannesBuchner]

Possibly related: https://en.wikipedia.org/wiki/Prediction_interval#Bayesian_statistics

[JorisDeRidder]

Both PyMC and PyStan implement the PPD of new data, hence the reason I was assuming too quickly that PredictionBand was meant to do the same. It's absolutely fine that PredictionBand is a general QuantileBand generator, but I'm sure that PyMC/Stan users would find it clarifying if the Time Series Fitting example of UltraNest would show that it can serve both to generate a quantile band of the mean response as well as a prediction band for new observations.