search

strengejacke / sjstats

Effect size measures and significance tests
https://strengejacke.github.io/sjstats
189 stars 21 forks source link

singular model and `sjstats::eta_sq` and `sjstats::omega_sq` #37

Closed IndrajeetPatil closed 6 years ago

IndrajeetPatil commented 6 years ago

If the error model for aovlist objects is singular, then eta-squared and omega-squared functions don't work (only if partial = FALSE).

Any way to safeguard against this? Maybe just produce NAs? 


# checking if broom::tidy works
broom::tidy(x = stats::aov(
  formula = mpg ~ wt + qsec + Error(disp / am),
  data = dplyr::filter(mtcars, cyl == 6),
  na.action = na.omit
))
#> # A tibble: 5 x 7
#>   stratum term         df sumsq meansq statistic p.value
#>   <chr>   <chr>     <dbl> <dbl>  <dbl>     <dbl>   <dbl>
#> 1 disp    wt            1 0.135  0.135    NA      NA    
#> 2 disp:am wt            1 7.85   7.85     NA      NA    
#> 3 Within  wt            1 0.852  0.852     0.480   0.560
#> 4 Within  qsec          1 0.288  0.288     0.162   0.726
#> 5 Within  Residuals     2 3.55   1.77     NA      NA

# eta-squared
sjstats::eta_sq(
  model = stats::aov(
    formula = mpg ~ wt + qsec + Error(disp / am),
    data = dplyr::filter(mtcars, cyl == 6),
    na.action = na.omit
  ),
  ci.lvl = 0.95,
  partial = FALSE
)
#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular
#> Error in cbind_all(x): Argument 2 must be length 4, not 6

# omega-squared
sjstats::omega_sq(
  model = stats::aov(
    formula = mpg ~ wt + qsec + Error(disp / am),
    data = dplyr::filter(mtcars, cyl == 6),
    na.action = na.omit
  ),
  ci.lvl = 0.95,
  partial = FALSE
)
#> # A tibble: 4 x 5
#>   term  omegasq conf.low conf.high stratum
#>   <chr>   <dbl>    <dbl>     <dbl> <chr>  
#> 1 wt    -0.113        NA    NA     disp   
#> 2 wt     0.421        NA    NA     disp:am
#> 3 wt    -0.0639       NA     0.504 Within 
#> 4 qsec  -0.103        NA     0.425 Within

# partial eta-squared
sjstats::eta_sq(
  model = stats::aov(
    formula = mpg ~ wt + qsec + Error(disp / am),
    data = dplyr::filter(mtcars, cyl == 6),
    na.action = na.omit
  ),
  ci.lvl = 0.95,
  partial = TRUE
)
#> # A tibble: 4 x 5
#>   term  partial.etasq conf.low conf.high stratum
#>   <chr>         <dbl>    <dbl>     <dbl> <chr>  
#> 1 wt           0.0366       NA    NA     disp   
#> 2 wt           0.689        NA    NA     disp:am
#> 3 wt           0.194         0     0.640 Within 
#> 4 qsec         0.0751        0     0.564 Within

# partial omega-squared
sjstats::omega_sq(
  model = stats::aov(
    formula = mpg ~ wt + qsec + Error(disp / am),
    data = dplyr::filter(mtcars, cyl == 6),
    na.action = na.omit
  ),
  ci.lvl = 0.95,
  partial = TRUE
)
#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular

#> Warning in stats::aov(mformula, data = i): Error() model is singular
#> Error in cbind_all(x): Argument 2 must be length 4, not 6

Created on 2018-07-13 by the reprex package (v0.2.0).

strengejacke commented 6 years ago

How do I check for singularity?

IndrajeetPatil commented 6 years ago

I am actually not sure if this is the problem. Because, as seen in the example above, even if the model is singular, the stats::aov function still works, and so does broom::tidy.

Here is a look at the process tree-

1: sjstats::omega_sq(model = stats::aov(formula = mpg ~ wt + qsec + Error(disp/am), data = dplyr::filter(mtcars, cyl == 6), na.action = na.omit), ci.lvl = 0.95
2: es_boot_fun(model = model, type = "pomega", ci.lvl = ci.lvl, n = n)
3: dplyr::bind_cols(x, es[, -1])
4: cbind_all(x)

So this is the line where the function is failing- https://github.com/strengejacke/sjstats/blob/1ec2a2cb55e92de1452fba9988624840998051b1/R/eta_sq.R#L442

It's then worth having a look at what's going on here- https://github.com/strengejacke/sjstats/blob/1ec2a2cb55e92de1452fba9988624840998051b1/R/eta_sq.R#L426-L439

Will do some digging myself sometime this week, in case you don't manage to resolve this before.

strengejacke commented 6 years ago

I'm on holiday and won't get back to this before August.