Baseframe should provide an A/B test framework for new app features. In its simplest form, the app initalizes Baseframe with a list of feature flags currently being tested:
Baseframe then adds a before_request handler that examines session['ab'], creating a dictionary if required and adding all missing flags with a value of None (not part of A/B test), False (A group) or True (B group):
for flag, chance in self.ab_flags: # Default chance is 1
if flag not in session['ab']:
session['ab'][flag] = random.choice([True, False]) if randint(1, chance) == 1 else None
An after_request handler runs through session['ab'] again and removes all keys that are not in self.ab_choices, to remove flags that the app is no longer testing.
Within the app, any transaction that is based on the feature can read the A/B flag status from session['ab'][flag]. It is not possible to read A/B flags from the client-side unless flags are stored in a separate, unencrypted (but still signed) non-HTTPOnly cookie.
Baseframe should provide an A/B test framework for new app features. In its simplest form, the app initalizes Baseframe with a list of feature flags currently being tested:
For a high impact experimental feature, the app may want to limit the test to, say, 1 in 10 users:
Baseframe then adds a
before_request
handler that examinessession['ab']
, creating a dictionary if required and adding all missing flags with a value ofNone
(not part of A/B test),False
(A group) orTrue
(B group):An
after_request
handler runs throughsession['ab']
again and removes all keys that are not inself.ab_choices
, to remove flags that the app is no longer testing.Within the app, any transaction that is based on the feature can read the A/B flag status from
session['ab'][flag]
. It is not possible to read A/B flags from the client-side unless flags are stored in a separate, unencrypted (but still signed) non-HTTPOnly cookie.