Open nicholasjclark opened 3 months ago
Need to also update loo.mvgam
to use posterior predictions rather than the draws generated from Stan. This is better for getting a less biased view of model fit, particularly for highly flexible AR / RW models:
loo_hcs = function(x, ...){
x$series_names <- levels(x$obs_data$series)
logliks <- logLik(x,
linpreds = predict(x,
newdata = x$obs_data,
type = 'link',
summary = FALSE,
process_error = TRUE),
newdata = x$obs_data,
family_pars = mvgam:::extract_family_pars(x))
logliks <- logliks[,!apply(logliks, 2, function(x) all(!is.finite(x)))]
releffs <- loo::relative_eff(exp(logliks),
chain_id = sort(rep(1:x$model_output@sim$chains,
(NROW(logliks) /
x$model_output@sim$chains))))
loo::loo(logliks, r_eff = releffs, ...)
}
PSIS-LOO can be used to generated weighted expectations and predictions, valuable for asking what predictions we would get if high-leverage "outliers" were downweighted: https://paul-buerkner.github.io/brms/reference/loo_predict.brmsfit.html.