data-sync-user
commented
1 month ago ➤ Daniel Miranda commented:
Hi Genevieve Baldassano! After investigating the potential for A/B testing the Donate banner, here are my findings:
Our current implementation of the Donate banner, as a snippet, is not A/B testable. This is because the package we use only supports A/B testing at the Page level, not for snippets or fields that aren’t directly tied to a page.
At a minimum, we could add a "Donate Banner" field directly to the Homepage model, which would allow us to run a standard out-of-the-box A/B test on the homepage, comparing different versions of the banner.
However, this raises the question of how the banner should be handled on other pages across the site. Here are two immediate options:
- Display the banner only on the homepage: The simplest approach would be to remove the banner from all other pages, focusing the A/B test solely on the homepage.
- Keep the "control" Donate Banner on other pages: We could continue using our current site-wide implementation to display a “control” version of the banner on all other pages. However, interactions on these pages wouldn’t be tracked as part of the A/B test.
Another option we could consider (though I am still investigating) is:
Implementing custom utilities/event types: According to the[ Wagtail-AB-testing docs|https://github.com/wagtail-nest/wagtail-ab-testing?tab=readme-ov-file#implementing-custom-goal-event-types], it appears we can create a custom event type that could be triggered on any page via JavaScript (for example, a click on the Donate banner’s link).
If we can implement custom logic on the BasePage ( https://github.com/MozillaFoundation/foundation.mozilla.org/blob/TP1-100-remove-localized-in-blogcard/network-api/networkapi/wagtailpages/pagemodels/base.py#L32 ) model to determine which Donate banner to render by checking the cookies set by Wagtail A/B testing, we could effectively run a site-wide A/B test using the banner variants defined on the homepage.
This approach though would be quite specific to this use case. Tagging Mavis Ou and Rob DiVincenzo for their thoughts!
Context: We’d like to be able to A/B test different creative in the foundation.mozilla.org homepage banner leading up to EOY fundraising. In order to do so, we need to confirm we can use Wagtail’s A/B testing for this type of snippet. Right now it seems like we’re not able to set up an A/B test with the banner with our current implementation. Can this be investigated and updated if possible?
Request: Enable homepage banner to be a/b testable.
┆Issue is synchronized with this Jira Task