openaustralia / oaf

OpenAustralia Foundation
https://github.com/openaustralia/oaf#readme
4 stars 3 forks source link

Run test comparing alternative suggested donation amounts on the 1st of JuneD #224

Closed equivalentideas closed 8 years ago

equivalentideas commented 8 years ago

With the first run of PlanningAlerts donation emails, use half the subscribers to runs a test comparing the return from alternative donation amounts:

Steps to this:

equivalentideas commented 8 years ago

Done the first steps of this:

equivalentideas commented 8 years ago

Our aim with this donations push is to maintain the number of contributions and revenue from our end-of-financial-year drive in 2015, but scale it up with the increase in users.

We send our first email to PlanningAlerts users on June 11th to 25063 people. There were 10254 opens (40% open-rate) and 403 clicks (4% click through). We received $4069 before we sent our second email from 102 contributions (25% conversion rate on the page, .4% from total sent).

In this tests we're sending to 1900 people (2000 minus people who've opted-out) for each version of the suggested amounts.

If we scale down the results from the 2015 email, our comparison baseline is $320 from 8 contributions (.4% from total) from each group of 1900.

That's not very many people so these tests are going to many much statistical accuracy. But we might see if something is a disaster. We've also pushing for regular donors and have made considerable changes to the form.

I'm glad we're sending something to a smaller group before we send to everyone, but I'm not yet sure if we'll see much of an indication of which suggested amount set works better.

equivalentideas commented 8 years ago

I've extracted the review stage of this into a new issue so closing this one.