adding support for `htest` objects in `eta_squared` and friends

[IndrajeetPatil]

I am not sure if this is feasible, but it will be sweet if this is also supported.

oneway.test(extra ~ group, data = sleep)
#> 
#>  One-way analysis of means (not assuming equal variances)
#> 
#> data:  extra and group
#> F = 3.4626, num df = 1.000, denom df = 17.776, p-value = 0.07939

effectsize::eta_squared(model = oneway.test(extra ~ group, data = sleep))
#> Error in UseMethod("anova"): no applicable method for 'anova' applied to an object of class "htest"

^{Created on 2020-10-15 by the reprex package (v0.3.0.9001)}

[mattansb]

All htests are implamented in effectsize::effectsize():

(ow <- oneway.test(extra ~ group, data = sleep))
#> 
#>  One-way analysis of means (not assuming equal variances)
#> 
#> data:  extra and group
#> F = 3.4626, num df = 1.000, denom df = 17.776, p-value = 0.07939

effectsize::effectsize(ow)
#> Eta2 (partial) |       90% CI
#> -----------------------------
#>           0.16 | [0.00, 0.42]

^{Created on 2020-10-15 by the reprex package (v0.3.0)}

[IndrajeetPatil]

I am confused why this doesn't work in function environments then:

foo <- function() {
  # does the function work?
  print(mod <- stats::oneway.test(formula = extra ~ group, data = sleep))

  # extract effect sizes
  effectsize::eta_squared(mod)
}

# use the function
foo()
#> 
#>  One-way analysis of means (not assuming equal variances)
#> 
#> data:  extra and group
#> F = 3.4626, num df = 1.000, denom df = 17.776, p-value = 0.07939
#> Error in UseMethod("anova"): no applicable method for 'anova' applied to an object of class "htest"

Trace:

> traceback()
9: stats::anova(model)
8: is.data.frame(x)
7: colnames(model)
6: denDF %in% colnames(model)
5: .anova_es.anova(stats::anova(model), type = type, partial = partial, 
       generalized = generalized, ci = ci)
4: .anova_es.default(model, type = "eta", partial = partial, generalized = generalized, 
       ci = ci)
3: .anova_es(model, type = "eta", partial = partial, generalized = generalized, 
       ci = ci)
2: effectsize::eta_squared(mod) at #6
1: foo()

[IndrajeetPatil]

effectsize::effectsize and not effectsize::eta_squared!!!

Sorry about the confusion!

[mattansb]

haha no worries (:

[IndrajeetPatil]

One last thing: There is no way to customize output from effectsize::effectsize any further for htest objects, right? For example, changing the default effect size from eta to omega?

[mattansb]

Can do

Aov <- oneway.test(extra ~ group, data = sleep)

effectsize::effectsize(Aov)
#> Eta2 (partial) |       90% CI
#> -----------------------------
#>           0.16 | [0.00, 0.42]

effectsize::effectsize(Aov, es = "epsilon2")
#> Epsilon2 (partial) |       90% CI
#> ---------------------------------
#>               0.12 | [0.00, 0.37]

effectsize::effectsize(Aov, es = "omega2")
#> Omega2 (partial) |       90% CI
#> -------------------------------
#>             0.11 | [0.00, 0.36]

^{Created on 2020-10-15 by the reprex package (v0.3.0)}

[mattansb]

Is on dev (:

[IndrajeetPatil]

You rock!! Thanks a bunch.

[IndrajeetPatil]

Do you know why it fails for aovlist objects only? Even if I set es = "omega2", it still returns eta^2.

set.seed(123)
library(ez)
data(ANT)

mod <- 
  ezANOVA(
  data = ANT[ANT$error == 0, ],
  dv = rt,
  wid = subnum,
  within = .(cue, flank),
  between = group,
  detailed = TRUE,
  return_aov = TRUE
)
#> Warning: Collapsing data to cell means. *IF* the requested effects are a subset
#> of the full design, you must use the "within_full" argument, else results may be
#> inaccurate.

class(mod$aov)
#> [1] "aovlist" "listof"

effectsize::effectsize(mod$aov)
#> Group            |       Parameter | Eta2 (partial) |       90% CI
#> ------------------------------------------------------------------
#> subnum           |           group |           0.51 | [0.22, 0.69]
#> subnum:cue       |             cue |           0.97 | [0.95, 0.97]
#> subnum:cue       |       group:cue |           0.12 | [0.00, 0.24]
#> subnum:flank     |           flank |           0.99 | [0.98, 0.99]
#> subnum:flank     |     group:flank |           0.33 | [0.11, 0.49]
#> subnum:cue:flank |       cue:flank |           0.22 | [0.09, 0.31]
#> subnum:cue:flank | group:cue:flank |           0.26 | [0.12, 0.35]

effectsize::effectsize(mod$aov, es = "omega2", ci = 0.99)
#> Group            |       Parameter | Eta2 (partial) |       99% CI
#> ------------------------------------------------------------------
#> subnum           |           group |           0.51 | [0.06, 0.75]
#> subnum:cue       |             cue |           0.97 | [0.94, 0.98]
#> subnum:cue       |       group:cue |           0.12 | [0.00, 0.33]
#> subnum:flank     |           flank |           0.99 | [0.97, 0.99]
#> subnum:flank     |     group:flank |           0.33 | [0.03, 0.57]
#> subnum:cue:flank |       cue:flank |           0.22 | [0.04, 0.37]
#> subnum:cue:flank | group:cue:flank |           0.26 | [0.07, 0.41]

^{Created on 2020-10-15 by the reprex package (v0.3.0.9001)}

[mattansb]

Yes, because effectsize() was poorly designed catch-all function, and I encourage people not to use it because it is not as explicit as the other functions 😅

I'll try and make it more robust, but I suggest using this only for htest objects, which is the only new thing offered by this function.

[mattansb]

From the function description: This function tries to return the best effect-size measure for the provided input model.

I think over crowding this with options is a bad move. This should be the "I don't know what I'm doing function". If you're asking for omega, you do know what your doing.... no?

[IndrajeetPatil]

Hmm, my quest to stick to just one function to extract both eta and omega is borne out of concerns about parsimony since I am using these functions in my packages. I don't want to rely on eta_squared in one context but then use effectsize function in another, depending on the class of object, to extract the same effect size. That opens me up to potential inconsistencies across contexts.

But I understand that htest objects might be a pain to accommodate in the general versions of these functions, and so there might be no other option but to toggle between different functions.

[mattansb]

htest are so weird - the chose of what info each one has is so arbitrary....

Like, why does chisq.test have the data, but all of the others don't? Very odd....

[DominiqueMakowski]

no wonder there is no "r" in the word consistency

[mattansb]

🤪

-- Mattan S. Ben-Shachar, PhD student Department of Psychology & Zlotowski Center for Neuroscience Ben-Gurion University of the Negev The Developmental ERP Lab

On Thu, Oct 15, 2020, 18:13 Dominique Makowski notifications@github.com wrote:

no wonder there is no "r" in the word consistency

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/easystats/effectsize/issues/175#issuecomment-709393607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AINRP6BPUCS3GLAI5FBC7ODSK4GRLANCNFSM4SRRKNMA .