equivalence_test for effectsize tables

[mattansb]

I've added an equivalence_test method for effectsize tables (the results to eta/d/cramers v/F_to* etc...). These are solely based on the CIs.

(I've also added plotting functions in see).

library(effectsize)
library(magrittr)
library(see)

etas <- aov(mpg ~ factor(am) * factor(cyl), data = mtcars) %>% 
  eta_squared() %>% 
  equivalence_test(range = 0.3)
etas
#> # Test for Practical Equivalence
#> 
#>   ROPE: [0.00 0.30]
#> 
#> Parameter              | Eta_Sq_partial |       90% CI |       H0
#> -----------------------------------------------------------------
#> factor(am)             |           0.63 | [0.42, 0.75] | Rejected
#> factor(cyl)            |           0.66 | [0.45, 0.77] | Rejected
#> factor(am):factor(cyl) |           0.10 | [0.00, 0.27] | Accepted
plot(etas)

@DominiqueMakowski @strengejacke can you take a look at these and make sure there return the correct labels?

ds <- t_to_d(t = c(1,-1.3,-3,2.3),
             df_error = c(40,35,40,85)) %>% 
  equivalence_test(res1, range = 1)
ds 
#> # Test for Practical Equivalence
#> 
#>   ROPE: [-1.00 1.00]
#> 
#>     d |         95% CI |        H0
#> ----------------------------------
#>  0.32 | [-0.31,  0.94] |  Accepted
#> -0.44 | [-1.11,  0.23] | Undecided
#> -0.95 | [-1.60, -0.29] |  Rejected
#>  0.50 | [ 0.07,  0.93] | Undecided
plot(ds)

[strengejacke]

everything completely inside the ROPE is "accepted".

[mattansb]

Okay, so like this?

library(effectsize)

ds <- t_to_d(t = c(0.45, -0.65, -2.2, 2.25),
             df_error = c(675, 525, 900, 1875))

ds
#>     d |         95% CI
#> ----------------------
#>  0.03 | [-0.12,  0.19]
#> -0.06 | [-0.23,  0.11]
#> -0.15 | [-0.28, -0.02]
#>  0.10 | [ 0.01,  0.19]
(equi <- equivalence_test(ds, range = .2))
#> # Test for Practical Equivalence
#> 
#>   ROPE: [-0.20 0.20]
#> 
#>     d |         95% CI |        H0
#> ----------------------------------
#>  0.03 | [-0.12,  0.19] |  Accepted
#> -0.06 | [-0.23,  0.11] | Undecided
#> -0.15 | [-0.28, -0.02] |  Rejected
#>  0.10 | [ 0.01,  0.19] |  Accepted

plot(equi)

^{Created on 2020-04-23 by the reprex package (v0.3.0)}

[strengejacke]

Yes, that's at least the convention from kruschke that we follow in bayestestR and parameters.

[mattansb]

Awesome - thanks (:

[mattansb]

library(effectsize)
library(magrittr)

F_to_eta2(f = c(4.5,2,1,4), 
          df = 2,
          df_error = c(200,40,100,40)) %>% 
  equivalence_test() %>% 
  plot()

^{Created on 2020-04-23 by the reprex package (v0.3.0)}

[strengejacke]

ehm. wait. (still too tired). "rejected" only if 95% intervals are outside the rope. See e.g. https://easystats.github.io/see/articles/parameters.html#for-random-effects-1 and https://easystats.github.io/see/articles/bayestestR_files/figure-html/unnamed-chunk-23-1.png

[strengejacke]

If the HDI is completely outside the ROPE, the "null hypothesis" for this parameter is "rejected". If the ROPE completely covers the HDI, i.e., all most credible values of a parameter are inside the region of practical equivalence, the null hypothesis is accepted. Else, it’s undecided whether to accept or reject the null hypothesis.

[strengejacke]

And further:

If the full ROPE is used (i.e., 100% of the HDI), then the null hypothesis is rejected or accepted if the percentage of the posterior within the ROPE is smaller than to 2.5% or greater than 97.5%. Desirable results are low proportions inside the ROPE (the closer to zero the better).

(from ?bayestestR::equivalence_test).

[mattansb]

Hmmm... But it seems like that's not how the frequentist version works (I've read up some more in the last hour).

Due to the NHST logic:

• If P(data | H_x) < α
• Then reject H_x

Using CIs, this can be reframed as:

• If CI_1-α does not contain the value expected by H_x
• Then reject H_x

And so, with frequientist equivalence testing: If the null was rejected, and the SESOI was not, then we conclude that H0 is rejected.

Or decision matrix then looks like this:

	Halt rejected	Not rejected
Hnull rejected	Accept null*	Reject null
Not rejected	Accept null	Undecided

* Even though our result is significantly larger than 0, it is also significantly smaller than SESOI. So we accept the null, as it is defined by the ROPE.

From Laken's paper:

(Even when accepting the null, the Bayesians and frequentist methods differ... I agree that the Bayesian way is more elegant here, because we can look at the % of the posterior etc...)

[mattansb]

I see that in parameters the %inRope is determined by the % of the length of the range of the CI... This is trying to put Bayesian ideas in a frequentist method... I think that should be dropped, and only keep the labels, using the frequentist rules.

[mattansb]

library(effectsize)

ds <- t_to_d(t = c(0.45, -0.65, -2.2, 2.25, 7),
             df_error = c(675, 525, 900, 1875, 2000),
             ci = 0.95)

equivalence_test(ds, range = 0.2)
#> # Test for Practical Equivalence
#> 
#>   ROPE: [-0.20 0.20]
#> 
#>     d |         95% CI |        H0
#> ----------------------------------
#>  0.03 | [-0.12,  0.19] |  Accepted
#> -0.06 | [-0.23,  0.11] | Undecided
#> -0.15 | [-0.28, -0.02] |  Rejected
#>  0.10 | [ 0.01,  0.19] |  Accepted
#>  0.31 | [ 0.22,  0.40] |  Rejected

equivalence_test(ds, range = 0.2, rule = "cet")
#> # Conditional Test for Practical Equivalence
#> 
#>   ROPE: [-0.20 0.20]
#> 
#>     d |         95% CI |        H0
#> ----------------------------------
#>  0.03 | [-0.12,  0.19] |  Accepted
#> -0.06 | [-0.23,  0.11] | Undecided
#> -0.15 | [-0.28, -0.02] |  Rejected
#>  0.10 | [ 0.01,  0.19] |  Rejected
#>  0.31 | [ 0.22,  0.40] |  Rejected

equivalence_test(ds, range = 0.2, rule = "bayes")
#> # Test for Practical Equivalence
#> 
#>   ROPE: [-0.20 0.20]
#> 
#>     d |         95% CI |        H0
#> ----------------------------------
#>  0.03 | [-0.12,  0.19] |  Accepted
#> -0.06 | [-0.23,  0.11] | Undecided
#> -0.15 | [-0.28, -0.02] | Undecided
#>  0.10 | [ 0.01,  0.19] |  Accepted
#>  0.31 | [ 0.22,  0.40] |  Rejected
#> 
#> (Using Bayesian guidlines)

^{Created on 2020-04-23 by the reprex package (v0.3.0)}