search

ietf-tools / www

A customized CMS for the IETF website
BSD 3-Clause "New" or "Revised" License
25 stars 43 forks source link

add A/B testing capability #173

Open ghwood opened 2 years ago

ghwood commented 2 years ago

It would be helpful to be able to conduct A/B (aka "split") testing on certain web pages.

There seem to be a few options for implementing this within Wagtail, with Wagtail Experiments being one.

holloway commented 2 years ago

Hi @ghwood what kinds of things would you want to test?

ie, with A/B testing often it's about measuring some outcome in order to evaluate success of A or B. What are we measuring?

ghwood commented 2 years ago

Hi @Holloway A few initial conversion goals are being considered:

holloway commented 2 years ago

@ghwood thanks, and so you'd want to A/B test UI differences (eg button colours) and how they affect completion of registrations, etc?

ghwood commented 2 years ago

Yes. CTAs, copy, colors, etc.

ali-haghnia commented 2 years ago

@AndrewCalderSpringload 's recommendation for AB testing is wagtail-ab-testing. With our initial investigations, seems it can address all our needs.

CC: @holloway

mgax commented 10 months ago

A few initial conversion goals are being considered:

  • meeting registrations
  • endowment donations
  • email list subscriptions I expect these will be refined.

Out of the box, wagtail-ab-testing can track page visits within the site as conversion goals. This might mean, for example, that a user visits the https://www.ietf.org/endowment/donate-ietf-endowment/ page, regardless of whether they actually make a donation.

More useful would be to track clicks on links and buttons within the website, which may navigate the user to another website, and for the conversion goals above, it's reasonably straightforward. @ghwood, if the list is up-to-date and complete, I can implement them.

Tracking conversion goals on a different website, e.g. meeting registrations, is another matter: it requires cooperation from the other website and seems out of scope for this project.

ghwood commented 10 months ago

Yes, this list is generally current for conversions that are clicking on links to another website:

1) initiate meeting registration -> click on a link in a Wagtail page to visit https://registration.ietf.org 2) make a donation - > successfully submit the form at https://www.ietf.org/endowment/donate-ietf-endowment/ 3) email list subscriptions -> click a link to visit a webpage at https://www.ietf.org/mailman/listinfo/* (where the exact webpage would depend on the list. Let me know if I can provide any further detail about any of these.