strengejacke / sjstats

Effect size measures and significance tests
https://strengejacke.github.io/sjstats
189 stars 21 forks source link

sjstats::omega_sq not working when scaled variables are used #27

Closed IndrajeetPatil closed 6 years ago

IndrajeetPatil commented 6 years ago

Below I am providing a fully reproducible example of an instance where sjstats::omega_sq is not working properly. There are two issues:

  1. Lower bound for CI is NA for omega-squared. (But maybe it is not defined in this case? The same issue also observed with function from another package. So this is not specific to sjstats.)
  2. When variables are scaled, partial omega-squared is not obtained

The same issue was not observed with the function partial eta-squared.

library(sjstats)

# simple model
mod1 <- stats::lm(formula = Sepal.Length ~ Sepal.Width,
                  data = datasets::iris)

# omega-squared
sjstats::omega_sq(
  model = mod1,
  partial = FALSE,
  ci.lvl = 0.95,
  n = 100
)
#> # A tibble: 1 x 4
#>   term        omegasq conf.low conf.high
#>   <chr>         <dbl>    <dbl>     <dbl>
#> 1 Sepal.Width 0.00711       NA    0.0717

# partial omega-squared
sjstats::omega_sq(
  model = mod1,
  partial = TRUE,
  ci.lvl = 0.95,
  n = 100
)
#> # A tibble: 1 x 4
#>   term        partial.omegasq conf.low conf.high
#>   <chr>                 <dbl>    <dbl>     <dbl>
#> 1 Sepal.Width         0.00711  -0.0223    0.0580

# the same model with scaled variables
mod2 <-
  stats::lm(formula = scale(Sepal.Length) ~ scale(Sepal.Width),
            data = iris)

# omega-squared
sjstats::omega_sq(
  model = mod2,
  partial = FALSE,
  ci.lvl = 0.95,
  n = 100
)
#> # A tibble: 1 x 4
#>   term               omegasq conf.low conf.high
#>   <chr>                <dbl>    <dbl>     <dbl>
#> 1 scale(Sepal.Width) 0.00711       NA    0.0717

# partial omega-squared
sjstats::omega_sq(
  model = mod2,
  partial = TRUE,
  ci.lvl = 0.95,
  n = 100
)
#> Error in mutate_impl(.data, dots): Evaluation error: object 'Sepal.Length' not found.

# getting confidence interval from userfriendlyscience package
# the same issue
userfriendlyscience::confIntOmegaSq(
  var1 = iris$Sepal.Length,
  var2 = iris$Sepal.Width,
  conf.level = 0.95
)
#> Omega squared: 95% CI = [NA; .16], point estimate = .03

Created on 2018-04-15 by the reprex package (v0.2.0).

strengejacke commented 6 years ago

First issue comes from MBESS::conf.limits.ncf(), which returns NA, same as userfriendlyscience::confIntOmegaSq() does. I did not dive deeper into the code to check where this issue may occur, but I'd say it's nothing I can fix in sjstats.

Second issue is fixed, but I strongly recommend doing data and variable transformation before modelling and not include these functions in the formula. Furthermore, scale() returns a matrix and no vector - maybe you can use sjmisc::std() or sjmisc::center() instead? I know some packages - inlcluding my own - where it causes difficulties if formulas contain functions like scale() or as.factor() etc., and as developer it's a mess to consider all these things ;-)