zalando / expan

Open-source Python library for statistical analysis of randomised control trials (A/B tests)
MIT License
335 stars 50 forks source link

Sample size with an unequal split ratio #187

Closed jbao closed 6 years ago

jbao commented 6 years ago

Hi all,

I'm wondering in the sample size formula,

https://github.com/zalando/expan/blob/master/expan/core/statistics.py#L168

whether the number of variants n should be replaced by 1+r? Esp. considering the current formula will give a smaller sample size with an increasing split ratio r, ie. an unequal split would require a smaller sample size to reach the same error rate, which is not so intuitive?

Thanks!

gbordyugov commented 6 years ago

Please discuss and review

https://github.com/zalando/expan/pull/188