Closed Daiki-Nakamura-git closed 1 week ago
Regarding $dz$ and $d{rm}$ - those differences in the CIs are rather small, and are the result of the different methods (ncp vs normal approximation). I don't think we will be able to have all functions across the package consistently use the same CI method - some methods are more well defined for some methods compared to others.
Regarding the supposedly larger deviations between $db$ vs. Glass's $\Delta$ / $d{av}$ vs Unpooled-SD d / "just d" vs d: It would be wrong for the former CIs to match the latter CIs since the repeated measures variants account for the fact that samples are nested within person, and so they should be more narrow (we are more certain about the true value of the effect size).
Bellow I demonstrate this with a bootstrap simulation:
The slabs are the bootstrapped sampling distribution of the effect size (top - bootstrap accounting for nesting; bottom - bootstrap not accounting for nesting). The black spikes mark the CIs as returned by the various {effectsize}
functions.
We can see:
rm_d()
variant are more narrow than the cohens_d()
/ glass_delta()
, {effectsize}
functions.Thanks for your reply.
My point about d_b, d_av, and "just d" was incorrect, as your simulation results show. I did not take the correlations into account.
I would prefer to give users a choice of which CI method to use for d_z and d_rm, but I can respect your idea of using different methods for different functions.
The confidence intervals for the effect size of repeated measures calculated by the rm_d() function do not match the results of other existing functions. For example, the following two outputs have matching point estimates but not matching confidence intervals.
The reason for the discrepancy is that rm_d() calculates confidence intervals by approximating the sample distribution with a normal distribution, while the other existing functions calculate confidence intervals using the non-central distribution method. I consider the non-central distribution method to be more accurate.
In addition, the results of the following functions should be consistent.
I would like the confidence intervals to be calculated in the same method consistently within the package.