`estimate_contrasts()` ignoring `adjust` argument

[dtgnn]

Hi and thank you for your work on easystats. I wanted to point out that the contrast estimation function appears to handle the adjust argument incorrectly.

Changing the adjustment method has no effect on the p-values. A comparison with emmeans also shows that the default method used by modelbased is actually "tukey" instead of "holm", despite what is written in the documentation and what is printed in the footer of the output table:

data("penguins", package = "palmerpenguins")

lm_fit <- lm(body_mass_g ~ species, data = penguins)

modelbased::estimate_contrasts(lm_fit)
#> Registered S3 methods overwritten by 'parameters':
#>   method                           from      
#>   as.double.parameters_kurtosis    datawizard
#>   as.double.parameters_skewness    datawizard
#>   as.double.parameters_smoothness  datawizard
#>   as.numeric.parameters_kurtosis   datawizard
#>   as.numeric.parameters_skewness   datawizard
#>   as.numeric.parameters_smoothness datawizard
#>   print.parameters_distribution    datawizard
#>   print.parameters_kurtosis        datawizard
#>   print.parameters_skewness        datawizard
#>   summary.parameters_kurtosis      datawizard
#>   summary.parameters_skewness      datawizard
#> Registered S3 method overwritten by 'modelbased':
#>   method                     from      
#>   print.visualisation_recipe datawizard
#> Marginal Contrasts Analysis
#> 
#> Level1    |    Level2 | Difference |               95% CI |    SE | t(339) |      p
#> -----------------------------------------------------------------------------------
#> Adelie    | Chinstrap |     -32.43 | [ -194.85,   130.00] | 67.51 |  -0.48 | 0.881 
#> Adelie    |    Gentoo |   -1375.35 | [-1510.44, -1240.27] | 56.15 | -24.50 | < .001
#> Chinstrap |    Gentoo |   -1342.93 | [-1511.00, -1174.86] | 69.86 | -19.22 | < .001
#> 
#> Marginal contrasts estimated for species
#> p-value adjustment method: Holm (1979)
modelbased::estimate_contrasts(lm_fit, adjust = "tukey")
#> Marginal Contrasts Analysis
#> 
#> Level1    |    Level2 | Difference |               95% CI |    SE | t(339) |      p
#> -----------------------------------------------------------------------------------
#> Adelie    | Chinstrap |     -32.43 | [ -191.35,   126.50] | 67.51 |  -0.48 | 0.881 
#> Adelie    |    Gentoo |   -1375.35 | [-1507.53, -1243.18] | 56.15 | -24.50 | < .001
#> Chinstrap |    Gentoo |   -1342.93 | [-1507.38, -1178.48] | 69.86 | -19.22 | < .001
#> 
#> Marginal contrasts estimated for species
#> p-value adjustment method: Tukey
modelbased::estimate_contrasts(lm_fit, adjust = "sidak")
#> Marginal Contrasts Analysis
#> 
#> Level1    |    Level2 | Difference |               95% CI |    SE | t(339) |      p
#> -----------------------------------------------------------------------------------
#> Adelie    | Chinstrap |     -32.43 | [ -194.43,   129.57] | 67.51 |  -0.48 | 0.881 
#> Adelie    |    Gentoo |   -1375.35 | [-1510.09, -1240.62] | 56.15 | -24.50 | < .001
#> Chinstrap |    Gentoo |   -1342.93 | [-1510.56, -1175.30] | 69.86 | -19.22 | < .001
#> 
#> Marginal contrasts estimated for species
#> p-value adjustment method: Sidak

emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "holm")$contrast
#>  contrast           estimate   SE  df t.ratio p.value
#>  Adelie - Chinstrap    -32.4 67.5 339  -0.480 0.6313 
#>  Adelie - Gentoo     -1375.4 56.1 339 -24.495 <.0001 
#>  Chinstrap - Gentoo  -1342.9 69.9 339 -19.224 <.0001 
#> 
#> P value adjustment: holm method for 3 tests
emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "tukey")$contrasts
#>  contrast           estimate   SE  df t.ratio p.value
#>  Adelie - Chinstrap    -32.4 67.5 339  -0.480 0.8807 
#>  Adelie - Gentoo     -1375.4 56.1 339 -24.495 <.0001 
#>  Chinstrap - Gentoo  -1342.9 69.9 339 -19.224 <.0001 
#> 
#> P value adjustment: tukey method for comparing a family of 3 estimates
emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "sidak")$contrasts
#>  contrast           estimate   SE  df t.ratio p.value
#>  Adelie - Chinstrap    -32.4 67.5 339  -0.480 0.9499 
#>  Adelie - Gentoo     -1375.4 56.1 339 -24.495 <.0001 
#>  Chinstrap - Gentoo  -1342.9 69.9 339 -19.224 <.0001 
#> 
#> P value adjustment: sidak method for 3 tests

^{Created on 2021-09-13 by the reprex package (v2.0.0)}

Session info

``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os macOS High Sierra 10.13.6 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz America/New_York #> date 2021-09-13 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> backports 1.2.1 2020-12-09 [1] CRAN (R 4.0.2) #> bayestestR 0.11.0 2021-09-03 [1] CRAN (R 4.0.2) #> cli 3.0.1 2021-07-17 [1] CRAN (R 4.0.2) #> coda 0.19-4 2020-09-30 [1] CRAN (R 4.0.2) #> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.0.2) #> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.2) #> datawizard 0.2.0.1 2021-09-02 [1] CRAN (R 4.0.2) #> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2) #> effectsize 0.4.5 2021-05-25 [1] CRAN (R 4.0.2) #> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.2) #> emmeans 1.6.1 2021-06-01 [1] CRAN (R 4.0.2) #> estimability 1.3 2018-02-11 [1] CRAN (R 4.0.1) #> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0) #> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.0.2) #> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2) #> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [1] CRAN (R 4.0.2) #> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.0.2) #> insight 0.14.4 2021-09-02 [1] CRAN (R 4.0.2) #> knitr 1.33 2021-04-24 [1] CRAN (R 4.0.2) #> lattice 0.20-44 2021-05-02 [1] CRAN (R 4.0.2) #> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.2) #> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.2) #> MASS 7.3-54 2021-05-03 [1] CRAN (R 4.0.2) #> Matrix 1.3-4 2021-06-01 [1] CRAN (R 4.0.2) #> modelbased 0.7.0 2021-06-06 [1] CRAN (R 4.0.2) #> multcomp 1.4-17 2021-04-29 [1] CRAN (R 4.0.2) #> mvtnorm 1.1-2 2021-06-07 [1] CRAN (R 4.0.2) #> palmerpenguins 0.1.0 2020-07-23 [1] CRAN (R 4.0.2) #> parameters 0.14.0 2021-05-29 [1] CRAN (R 4.0.2) #> pillar 1.6.2 2021-07-29 [1] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0) #> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0) #> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.0.0) #> Rcpp 1.0.7 2021-07-07 [1] CRAN (R 4.0.2) #> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.0.2) #> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.0.2) #> rmarkdown 2.10 2021-08-06 [1] CRAN (R 4.0.2) #> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.2) #> sandwich 3.0-1 2021-05-18 [1] CRAN (R 4.0.2) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.2) #> stringi 1.7.3 2021-07-16 [1] CRAN (R 4.0.2) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0) #> styler 1.5.1 2021-07-13 [1] CRAN (R 4.0.2) #> survival 3.2-11 2021-04-26 [1] CRAN (R 4.0.2) #> TH.data 1.0-10 2019-01-21 [1] CRAN (R 4.0.2) #> tibble 3.1.3 2021-07-23 [1] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.0.2) #> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.0.2) #> withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.2) #> xfun 0.25 2021-08-06 [1] CRAN (R 4.0.2) #> xtable 1.8-4 2019-04-21 [1] CRAN (R 4.0.0) #> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0) #> zoo 1.8-9 2021-03-09 [1] CRAN (R 4.0.2) #> #> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library ```

[profandyfield]

Just wanted to add a +1. This has broken a load of my learnr tutorials (in that text no longer matches output). Here's another reproducible example:

pupluv_tib <- read.csv("https://www.discovr.rocks/csv/puppy_love.csv") |>
  dplyr::mutate(
    dose = forcats::as_factor(dose)
  )

pupluv_lm <- lm(happiness ~ puppy_love + dose, data = pupluv_tib)

modelbased::estimate_contrasts(pupluv_lm, fixed = "puppy_love", adjust = "holm")

modelbased::estimate_contrasts(pupluv_lm, fixed = "puppy_love", adjust = "bonferroni")