search

ethen8181 / machine-learning

:earth_americas: machine learning tutorials (mainly in Python3)
MIT License
3.19k stars 650 forks source link

Possible mistake in sanity check function #12

Closed dbalabka closed 3 years ago

dbalabka commented 3 years ago

z variable isn't used in interval calculation:

def sanity_check(size1, size2, significance = 0.05):
    n = size1 + size2
    confidence = 1 - significance
    z = stats.norm.ppf(confidence + significance / 2)
    confint = n * 0.5 + np.array([-1, 1]) * np.sqrt(n * 0.5 * 0.5)
    return confint

Source: http://ethen8181.github.io/machine-learning/ab_tests/frequentist_ab_test.html#Sanity-Check

ethen8181 commented 3 years ago

Thanks for pointing this out, I've added the multiplication of the z variable in the confidence interval calculation with the latest commit. https://github.com/ethen8181/machine-learning/commit/39d5d02a79840e8c694bedb427c575d0bfe7ed5f

ethen8181 commented 3 years ago

closing this as it's resolved, feel free re-open if you feel that's not the case.