Alternative Gaussian process surrogate based calibration approaches

[matt-graham]

Some initial ideas of additional calibration approaches to try out that use a similar approach to variational Bayesian Monte Carlo (VBMC) of fitting a Gaussian process surrogate to evaluations of the log posterior density / likelihood:

• No-Regret Approximate Inference via Bayesian Optimisation (Oliveira, Ott and Ramos; UAI 2021) - 📄 paper, 💻 code
  • Uses Markov chain Monte Carlo based approximation of surrogate posterior to choose new points to evaluate model at.
  • Seems to potentially give improved performance on for example 'ridged' density that VBMC Gaussian mixture variational approximation requires many mixture components to fit well, though also less robust to multimodality and reference implementation less easy adapt to new problems.
  • Would be interesting to instead try out a sequential Monte Carlo based approach, which might help improved robustness to multimodality and also be better suited to distributing computations.
  • Parallel Gaussian Process Surrogate Bayesian Inference with Noisy Likelihood Evaluations (Järvenpää, Gutmann, Vehtari and Marttinen) 📄 paper, 💻 code
• Proposes batch Bayesian optimisation / sequential design approach to select multiple candidate points to evaluate model at in parallel.
• Inherently designed for models which we can only evaluate a noisy likelihood estimate.

[matt-graham]

The Parallel Gaussian Process Surrogate Bayesian Inference with Noisy Likelihood Evaluations approach has been implemented in a new package calibr. We didn't get as far as applying this package to NESO models in the AQUIFER project timeline but we may revisit this in future if we continue with development of calibr.