search

CamDavidsonPilon / Probabilistic-Programming-and-Bayesian-Methods-for-Hackers

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)
http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/
MIT License
26.55k stars 7.85k forks source link

Online machine learning for a/b testing #394

Closed paulpach closed 6 years ago

paulpach commented 6 years ago

I would love to see a chapter on how to do bayesian online machine learning with a/b testing as described here: https://americanairlines.udemy.com/bayesian-machine-learning-in-python-ab-testing/learn/v4/overview

sorry for the paywall, I will try to explain it to the best of my abilities (I am no data scientist):

Basically the goal is to do A/B/C/... testing online. It means we adjust the model for every single new sample and over time the system automatically offers more of the winning option, and reduces exploration of less performing options.

For example, let's say we have 3 ads A,B,C. Lets say A has a click through rate of 1%, B has a CTR of 1.5% and C has a CTR of 5%.

When we launch the site, we don't know the CTR of these ads, we want to test them and automatically pick the best one as we learn more about them.

1) We assume they all have the same CTR in the prior distribution. we can just take a guess of 1% uniform distribution CTR for all 3, the initial value is not that important. 2) When a user visits the site, we sample all 3 distributions, and pick the one that returned the highest CTR. 3) We display the ad that returned the highest CTR 4) The user will click on it or not. Now we have 1 observed sample for that particular ad, we update the model for that ad's distribution. 5) we go back to step 2

This continues for ever. Note that as the algorithm learns more and more, it will tend to pick ad 3 more and more, so the algorithm will get more and more confidence on 3's CTR. But ad 1 and 2, will not be chosen as much, it is a waste trying to pick them because at some point ad 3 will be clearly performing better. ad 1 and 2 CTR distribution will remain flatter while ad 3 will get narrower and narrower.

CamDavidsonPilon commented 6 years ago

Hey, see chapter 6 on Priors - this examples (or similar to it) is in there.