search

ACCLAB / DABEST-python

Data Analysis with Bootstrapped ESTimation
https://acclab.github.io/DABEST-python/
Apache License 2.0
341 stars 47 forks source link

Show bootstrap 95% confidence interval for Control? #54

Closed FinweNoldoran closed 5 years ago

FinweNoldoran commented 5 years ago

As the title suggests, I'd like to show the bootstrap 95% confidence interval for the control group too, is there a way of doing that?

Thanks

adamcc commented 5 years ago

Hi! I'm not sure I understand your question. The effect size CI and curve is always calculated from the difference between a test group and the control group.

(By definition, the effect size of the control against itself is always zero with a zero-width CI, so we don't show it in the multi-group plots. In earlier versions we showed it as a black dot on zero)

In some plots, we show error bars next to the observed values, but these are standard deviations, not CIs.

FinweNoldoran commented 5 years ago

Hi, I'm not really clued up on this yet, so please excuse my ignorance.

When I have a control group, there is distribution. The swarm plot shows the standard deviation of this. Is it not possible to calculate a CI for the control and plot it on the same axis as the effect size CI (maybe that is a really stupid question, sorry)?

When you plot the effect size below, especially in the multi-group plots, it reads naively that the control group has no distribution or error associated with it.

What I'm trying to say, perhaps poorly, is that, say you have a control and a test sample that are not significantly different. In the swarm plot, this is totally obvious. However, when plotting the effect size with CI, the CI and curve may not overlap with a mean difference of 0, which one might naively read as there actually being a significant difference in the samples. So if there was a way of showing the uncertainty of the control on that axis too, I think that would be helpful.

I hope that makes some sense.

adamcc commented 5 years ago

Hi there.

The effect-size curve is a model of the difference between the control and test groups. As such, it incorporates variance from both control and test. Since both variances are baked into the effect-size curve, the two visual guidelines to precision are (1) the narrowness of the curve, and (2) its separation from the zero-effect line.

In the first column of the shared-control plot, there is an empty slot. In the example data, this would read "A1 minus A1." The slot is left empty because subtracting the control group from itself has no rationale and/or would be zero (with zero variance).

Screenshot 2019-08-05 at 9 11 30 PM
FinweNoldoran commented 5 years ago

Thanks, I think I understand now!

adamcc commented 5 years ago

👍

josesho commented 5 years ago

Closing this as no further action is required. Thanks!