search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
58 stars 29 forks source link

Adding non-parametric effect size estimators #450

Open chantelanuit opened 5 years ago

chantelanuit commented 5 years ago
* Enhancement: Easy to understand and easy to compare non-parametric effect size measures ("probability of superiority") are lacking in most statistical analysis software * Purpose: **Include state-of-the-art probability-based effect size indicators** (e.g. "probability that a randomly selected member of one group scores higher than a random selected member of another group") + confidence interval around the effect size * Use-case: **Is your feature request related to a problem? Please describe.** The most commonly used effect-size measures (e.g., Cohen's d) require several assumptions to be valid (normality, homoscedasticity, no outliers, ...). Several other measures are available but unfortunately not included in most statistical software. One of the best options available is Vargha & Delaney A (2000) and generalizations of this measure. As stated by Ruscio (2013, p. 210): > "The A statistic does not require parametric assumptions, is highly robust to the influence of outliers, and is insensitive to unequal group sizes— which means that it can be more helpful when generalizing findings to other research contexts. Perhaps most important is that A is easier to understand than d or rpb, facilitating communication even with those untutored in statistics." The A statistic is relevant for both parametric and non-parametric tests and is a great step toward easing the comparison between these tests. **Describe the solution you'd like** Integrating the output of the effisize (https://cran.r-project.org/web/packages/effsize/index.html) and the RProbSup (https://cran.r-project.org/web/packages/RProbSup/index.html) packages in JASP. The later package is specific to the A measures and includes confidence intervals for the effect size. For additional information, please see: Ruscio, J., & Gera, B. L. (2013). Generalizations and Extensions of the Probability of Superiority Effect Size Estimator. Multivariate Behavioral Research, 48(2), 208-219. doi:10.1080/00273171.2012.738184 **Describe alternatives you've considered** **Additional context**
EJWagenmakers commented 5 years ago

Another good suggestion!

chantelanuit commented 4 years ago

To add to my previous post, see: Chao-Ying Joanne Peng & Li-Ting Chen (2014) Beyond Cohen's d: Alternative Effect Size Measures for Between-Subject Designs, The Journal of Experimental Education, 82:1, 22-50, DOI: 10.1080/00220973.2012.745471

"Of the nine estimators summarized in Table 2b, we recommend the four estimators of dominance in Category (B) to supplement Cohen’s d to conceptualize ES beyond mean differences. Of these four estimators, Vargha and Delaney’s  stands out for its meaningful interpretability in terms of stochastic equality/superiority or stochastic homogeneity/heterogeneity in a variety of research contexts and for a variety of data types. Compared to Cohen’s d, Vargha and Delaney’s  represents a radical reconceptualization of ES with sound statistical properties and well developed theoretical framework." (p. 45) Source: https://www.tandfonline.com/doi/abs/10.1080/00220973.2012.745471

tomtomme commented 8 months ago

Related requests for more effect size measures: