JustFixNYC / tenants2

The JustFix tenant platform!
https://app.justfix.org/
GNU General Public License v3.0
23 stars 4 forks source link

Add support for A/B testing #562

Open toolness opened 5 years ago

toolness commented 5 years ago

It would be nice if this project easily supported A/B testing.

I found a blog post on How to quickly set up A/B testing for React websites and after skimming it, I think we could probably follow something like its strategy, so long as we stored the user's variant information in their session rather than localStorage, since we want server-side rendering to be properly synced with the client.

Regarding the actual service we use to measure the conversions, I think the most straightforward way to implement it without adding new third-party services would be to store the user's variant information in a GA custom dimension. This would then allow us to generate reports in GA that took the user's variant information into account, e.g. measuring transitions from the landing page to step 1 of onboarding segmented by variant.

Once we expose the variant on the client-side, we can also conditionally render based on it. That part seems pretty straightforward, at least insofar as we keep variant content in the project's codebase rather than on a third-party content management service.

(Relatedly, I've been playing with Optimizely's A/B Test Sample Size Calculator and the results are interesting.)

toolness commented 5 years ago

Research A/B testing options