easystats / performance

:muscle: Models' quality and performance metrics (R2, ICC, LOO, AIC, BF, ...)
https://easystats.github.io/performance/
GNU General Public License v3.0
1.03k stars 94 forks source link

Comparing frequentist and Bayesian models #738

Open roaldarbol opened 5 months ago

roaldarbol commented 5 months ago

Is it possible to compare a Bayesian and frequentist model in a meaningful way? I tried compare_performance() for comparing the same model specified with brm and glmmTMB, but the error suggests to me that it might not be possible (or meaningful). Error is:

> performance::compare_performance(
+   mod_ratio_state, 
+   bayes_ratio_state)
Error in loo::stacking_weights(lpd_point) : 
  At least two models are required for stacking weights.

(I'm trying to make a case for using Variance Ratio over ICC, and am trying to pre-empt questions about the model fits).

Might relate to https://github.com/easystats/performance/issues/716.