search

easystats / effectsize

:dragon: Compute and work with indices of effect size and standardized parameters
https://easystats.github.io/effectsize/
Other
334 stars 23 forks source link

Allow extra argument type=1, 2 or 3 for eta_squared function & change default to type=2 #526

Open tomwenseleers opened 1 year ago

tomwenseleers commented 1 year ago

Describe the solution you'd like The current eta_squared function appears to return partial eta squared values based on type I SS. It is generally reckoned those are not very useful for factorial designs or models with interaction effects. It also leads to confusion, as the partial eta squared values of different packages have different defaults & options in this respect, sometimes using type I SS and sometimes type II or III, https://stats.stackexchange.com/questions/552451/which-r-functions-are-correct-for-estimating-partial-eta-squared-for-effects-in. At the very least an extra argument type=XX would seem to be in order, as in the DescTools::EtaSq function.

How could we do it? To get partial eta squared values using type I, II or III SS I figured that with the effectsize::eta_squared function one would have to do effectsize::eta_squared(fit) or equivalently effectsize::eta_squared(anova(fit)) # using type I tests effectsize::eta_squared(car::Anova(fit, type=2)) # using type II tests effectsize::eta_squared(car::Anova(fit, type=3)) # using type III tests, requires using effect coding / sum contrasts, cf afex::set_sum_contrasts() and all continuous covariates need to be centered using scale()

It would be nice if these could be obtained using effectsize::eta_squared(fit, type=1, 2 or 3)

mattansb commented 1 year ago

This is definitely something I want to implement - I would have to see how to consistently do this across model types.

Thanks for the suggestion.

mattansb commented 1 year ago

This will require a re-write of a lot of code. Essentially we would need to have (internally) two types of processing functions:

  1. Those that deal with ANOVA tables as inputs. These DON'T not get a "type" argument, since that is a characteristic of the table.
  2. Those the deal with models as inputs. These DO get a "type" argument, and pass the resulting ANOVA table to the functions that deal with ANOVA tables as inputs (1).

For each model class, the "type" argument would have to be checked against what is possible for that model, and which packages are available. For example:

@strengejacke I know some of this is available in parameters - do you know what might be missing to that effectsize can completely rely on it?

strengejacke commented 1 year ago

You mean in terms of detecting the type?

mattansb commented 1 year ago

I think the type detection is fully implemented in parameters for ANOVA tables.

For aov / maov, what type is returned by parameters by default? Can I request all types? For Anova.mlm what is the default - univariate of multivariate tests?

tomwenseleers commented 1 year ago

If it would be too much hassle to rework large parts of the function, maybe just a little note in the help could be good saying that if you would like effect sizes for Anova tables (of whatever fit supported by https://rdrr.io/cran/car/man/Anova.html, i.e. lm, glm, multinom, polr, coxph, coxme, svyglm, svycoxph, rlm, lmer or lme), effectsize::eta_squared(car::Anova(fit, type=3)) is probably what you want (provided you used sum/effect coding using afex::set_sum_contrasts() and that all continuous covariates were centered first using scale())?

mattansb commented 1 year ago

Well, we already have that:

  1. In the function help page - an explanation about type 2/3 under "Type of Sums of Squares" section, and in the examples it says "Recommended: Type-2 or -3 effect sizes + effects coding", both with examples of use of car::Anova.
  2. The ANOVA vignette goes into great detail about using car::Anova or afex
tomwenseleers commented 1 year ago

Thanks - I only just noticed those bits now. I guess it comes down to it being a bit unfortunate that the R folks only implemented type I SSs in their default anova() method, which you then use for most models "For all other model, effect sizes are approximated via test statistic conversion of the omnibus F statistic provided by the appropriate anova() method". Whereas it's Anova(type="III") results one would typically like to see for any model. But as long as this caveat is pointed out somewhere & how the proper effect sizes using type III SSs can be obtained I'm fine with that... I just thought an extra argument type=1, 2 or 3 could make this a bit more explicit... (As this is also the way this is done in some other effect size packages)