vincentarelbundock / marginaleffects

R package to compute and plot predictions, slopes, marginal means, and comparisons (contrasts, risk ratios, odds, etc.) for over 100 classes of statistical and ML models. Conduct linear and non-linear hypothesis tests, or equivalence tests. Calculate uncertainty estimates using the delta method, bootstrapping, or simulation-based inference
https://marginaleffects.com
Other
470 stars 47 forks source link

'equivalence' argument for Bayesian models in avg_slopes function ? #879

Closed osupplisson closed 1 year ago

osupplisson commented 1 year ago

Hi,

I am currently estimating a bunch of average marginal effects and their (pairwise) between-group differences (diff-in-diff) using the avg_slopes function of the package and a brmsobject. It's amazing how easy it is, thank you !

For reporting completeness, I wanted to provide, along with the posterior average and ETI95%, the exceedance probability with respect to 0 (i.e., Pr(Marginal effect> 0 | Y) and Pr(Diff-in-Marginal effect> 0 | Y)).

The equivalence argument allows this kind of test for frequentist model but it causes an error for bayesian model (I guess you are aware of this because of issue https://github.com/vincentarelbundock/marginaleffects/issues/629):

library(brms)
library(marginaleffects)
fit1 <- brm(count ~ zAge, data = epilepsy, family = poisson())
avg_slopes(fit1 , variables = "zAge", equivalence = c(0))

Generating the following error:

Error: The equivalence argument is not supported with models for which marginaleffects does not estimate a standard error (e.g., bayesian).

Is there a workaround in the avg_slopes function for using this argument with bayesian models ?

Currently, I only see as fallback solution 'manual computation' of everything, which I would rather avoid... I am still discovering the package: I might miss something relevant.

vincentarelbundock commented 1 year ago

Thanks for the suggestion. I think it’s interesting, but I am wondering if it’s worth it. My sense is that one of the main advantages of bayesian computation is that a lot of these quantities are really trivial to compute. It would add code complexity to marginaleffects, and I’m not sure it would bring a ton of benefits relative to something like:

library(brms)
library(marginaleffects)
library(dplyr)
fit1 <- brm(count ~ zAge, data = epilepsy, family = poisson())

avg_slopes(fit1 , variables = "zAge") |>
    posterior_draws() |>
    summarize(estimate = mean(draw), low = mean(draw < 0), high = mean(draw > 0), .by = term)
osupplisson commented 1 year ago

Thank you very much for your answer.

I wasn't aware of the posterior_drawsfunction, which indeed makes it super easy to compute whatever I need, as you showed.