assaf / vanity

Experiment Driven Development for Ruby
http://vanity.labnotes.org
MIT License
1.55k stars 269 forks source link

Fix AbTest#alternative_for to always prefer persisted participants #371

Closed tgrathwell closed 2 years ago

tgrathwell commented 2 years ago

Prior to this fix, stubbing a specific outcome with AbTest#chooses would occasionally set the 'shown' bit on a participant because the code at https://github.com/assaf/vanity/blob/469917acf/lib/vanity/experiment/ab_test.rb#L242 noticed that the persisted participant's alternative differed from the alternative selected by the hashing algorithm in #alternative_for

This would cause problems downstream in the conversion code because #ab_showing returns true on this participant.

bensheldon commented 2 years ago

fyi, there is some more discussion of this in #333.

bensheldon commented 2 years ago

Released in v3.0.2 🎉