dancrew32 / ab

A/B Testing Framework for Python (with optional Multi-armed bandit implementation)
https://pypi.org/project/ab/
The Unlicense
3 stars 3 forks source link

Option to distinct user trials and successes for MAB #9

Open dancrew32 opened 5 years ago

dancrew32 commented 5 years ago

If one user creates all of the trials and successes, they can bias the experiment. As an alternative to incr on trials and successes, it may be useful to maintain redis sets and check the set length for each control/variant. This can also allow us to keep a user who succeeds with one variant pinned to that variant in future experiences.