search

Matt52 / bayesian-testing

Bayesian A/B testing
MIT License
72 stars 9 forks source link

Is Beta(1/2, 1/2) the right default for a non-information prior? #38

Closed johnmarcampbell closed 5 months ago

johnmarcampbell commented 5 months ago

https://github.com/Matt52/bayesian-testing/blob/516e9b6836aef5bbbca137459b16e4d307e930d5/bayesian_testing/experiments/binary.py#L125

Just a quick observation: if you want a flat Beta prior as the default, then based on numpy's parameterization of the Beta function I think Beta(1, 1) is the right choice.

Beta(1/2, 1/2) blows up at p=0 and p=1, and has a minimum at p=1/2. (See here.) If that's the behavior you want, please ignore this issue. But if you want a flat default then consider changing to Beta(1,1).

Cheers!

Matt52 commented 5 months ago

Hi John and thanks for your comment! In fact this is intended as I wanted to have a default prior to be a "non-information prior" using a Jeffreys prior. See for instance here: https://en.wikipedia.org/wiki/Jeffreys_prior#Bernoulli_trial In case you would like to use the uniform prior (a=b=1), you can set the prior parameters when adding variants.

Cheers!