easystats / effectsize

:dragon: Compute and work with indices of effect size and standardized parameters
https://easystats.github.io/effectsize/
Other
337 stars 23 forks source link

Appropriate to use t_to_d() on estimated marginal mean contrasts from ANOVA? #616

Closed DallasNovakowski closed 11 months ago

DallasNovakowski commented 11 months ago

Question and context

I'm running some factorial anova analyses, and would like to use t_to_d() with emmeans to retrieve cohen's d effect sizes for my pairwise contrasts. I'm just wondering whether this is statistically sound? I see the degrees of freedom are the same all across the contrasts, and am not sure whether these large df values might lead to some inflated and inaccurate estimates of cohen's d.

Reprex:

library(palmerpenguins) # dataset
#> Warning: package 'palmerpenguins' was built under R version 4.1.3
library(car) # anova functions
#> Warning: package 'car' was built under R version 4.1.3
#> Loading required package: carData
#> Warning: package 'carData' was built under R version 4.1.3
library(effectsize) # effect sizes
library(dplyr)
#> Warning: package 'dplyr' was built under R version 4.1.3
#> 
#> Attaching package: 'dplyr'
#> The following object is masked from 'package:car':
#> 
#>     recode
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

# assigns data to a dataframe we call "df"
df <- palmerpenguins::penguins

# drop rows with missing values
df <- df[complete.cases(df)==TRUE, ]

df %>%
    group_by(species, sex) %>% # Group by the specified variables
    dplyr::summarise(n()) %>%
  knitr::kable()
#> `summarise()` has grouped output by 'species'. You can override using the
#> `.groups` argument.
species sex n()
Adelie female 73
Adelie male 73
Chinstrap female 34
Chinstrap male 34
Gentoo female 58
Gentoo male 61

# Fit data
flipper_fit <- stats::aov(flipper_length_mm ~ species, data = df)
# Run anova
flipper_anova <- car::Anova(flipper_fit)

# Extract estimated marginal means
flipper_emmeans <- emmeans::emmeans(flipper_fit, specs = pairwise ~ species)

# convert estimated marginal mean contrasts to dataframe
flipper_emmeans_contrasts <- data.frame(flipper_emmeans$contrasts)

knitr::kable(flipper_emmeans_contrasts)
contrast estimate SE df t.ratio p.value
Adelie - Chinstrap -5.72079 0.9796493 330 -5.83963 0
Adelie - Gentoo -27.13255 0.8240767 330 -32.92479 0
Chinstrap - Gentoo -21.41176 1.0143492 330 -21.10887 0

effectsize::t_to_d(t = flipper_emmeans_contrasts[flipper_emmeans_contrasts$contrast == "Adelie - Chinstrap", "t.ratio"],
                   df_error = flipper_emmeans_contrasts[flipper_emmeans_contrasts$contrast == "Adelie - Chinstrap", "df"]
                   )
#> d     |         95% CI
#> ----------------------
#> -0.64 | [-0.86, -0.42]

Created on 2023-11-05 with reprex v2.0.2