vincentarelbundock / marginaleffects

R package to compute and plot predictions, slopes, marginal means, and comparisons (contrasts, risk ratios, odds, etc.) for over 100 classes of statistical and ML models. Conduct linear and non-linear hypothesis tests, or equivalence tests. Calculate uncertainty estimates using the delta method, bootstrapping, or simulation-based inference
https://marginaleffects.com
Other
455 stars 47 forks source link

`hypothesis_by` argument #1057

Closed vincentarelbundock closed 6 months ago

vincentarelbundock commented 6 months ago

How far does that take us into solving the hypothesis factory issues?

vincentarelbundock commented 6 months ago
k = matrix(rnorm(50), nrow = 10)
g = factor(rep(1:2, each = 5))
sta <- BY(k, g = g, FUN = function(x) x[1])
out <- TRA(k, g = g, FUN = "-", STATS = sta)
vincentarelbundock commented 6 months ago
    # if (is.matrix(draws)) {
    #     if (!is.null(hypothesis_by)) {
    #         g <- data.table(newdata)
    #         g <- as.list(g[, ..hypothesis_by])
    #         sta <- collapse::BY(draws, g = g, FUN = function(x) x[1])
    #         dr <- collapse::TRA(draws, g = g, FUN = "-", STATS = sta)
    #     } else {
    #         sta <- collapse::BY(draws, FUN = function(x) x[1])
    #         dr <- collapse::TRA(draws, FUN = "-", STATS = sta)
    #     }
    #     dr <- dr[idx,]
    #     attr(out, "posterior_draws") <- dr
    # }
    #
vincentarelbundock commented 6 months ago

The development version of marginaleffects on Github is not much more flexible in accepting custom functions in the hypothesis argument of all its functions. This is an extremely powerful strategy, but it can be tedious to create appropriate labels or apply functions by subgroups.

Thus, I have created a specify_hypothesis() function. This is a helper that returns a function which can then be passed to the hypothesis argument for complex aggregation or hypothesis testing. This is powerful, because it not only allows differences, but also any user-supplied function to compare estimates.

Admittedly, this still feels like a power user feature, but I like it because it's sooooo flexible. I'm keen to get feedback and suggestions on how to make this easier more intuitive. At least the machinery is all there now.

Tutorial here: https://marginaleffects.com/vignettes/hypothesis.html#functions

I recommend reading the "Functions" section from the top before getting to specify_hypothesis().

vincentarelbundock commented 4 months ago

In retrospect, I concluded that specify_hypothesis() is way too complicated. It was a mistake to export that function, and it will be deprecated.

Instead, I have implemented grouped hypothesis tests using an interface similar (but not identical) to the formula interface in emmeans.

I think it's pretty cool. Leaving link to the documentation here in case someone finds this issue: https://marginaleffects.com/vignettes/hypothesis.html#formulas-group-wise