General Data Questions --

[shaghdoosti]

Some questions I think people will ask during the review:

Which video had the most shares? Which video had the most views? Which subject line won? Which email had the highest click through rate? Did our open rates and CTR get higher with each email or did they fluctuate? Did views of our videos get higher with each beat or did they fluctuate? Did shares of our videos get higher with each beat or did they fluctuate? Which social media post had the most engagement? How did social media posts about the campaign (in terms of shares and comments) perform relative to other posts?

Please feel free to add others -- @stephaniemcv @HPaulJohnson would love your help to answer these.

[brettgaylor]

I think the P1 in this list is "active advocates", ie our KPI for the campaign. How many people took a repeated action?

[stephaniemcv]

Which subject line won? Highest Open Rate was Beat 3 - Can you read this message? at 46.51%. This higher rate is probably due to the fact that this was the first email after our deliverability issue and we had a very tight audience criteria of (All Active Advocates and General Foundation List subscribers from EN countries, who Opened Beat 1 or Kicker 1 or Beat 2)

Which email had the highest click through rate? Beat 5 had a Click Through Rate of 4.62%

Did our open rates and CTR get higher with each email or did they fluctuate? There was fluctuation in both OR and CTR. However, the general trend line is a positive increase as the campaign progresses. You can see all data and a nifty graph on the Total Tracking tab here: https://docs.google.com/a/mozilla.com/spreadsheets/d/17QiSIDWzs0WG9BqBYOfAacG59J2fuj_sSId2fNyO7QM/edit?usp=sharing

Active Advocate KPI

• The Active Advocate group imported from BSD is 143,935.
• We had 8,978 pledges from our Beat 5 email (This was queried by link clicks and modify date on current subscriber account)
• Per Adam's suggestion in ticket #130, I ran a query against this group to determine that 7,160 are not in the original Active Advocate group.
• We have 390 who are new subscribers (Source Url is landing page and Created Date is on/after 3/31/16)

[HPaulJohnson]

@shaghdoosti @brettgaylor @stephaniemcv @adamlofting Looking at our active advocates goal and where we are at based on these numbers, I'm wondering if there are optimizations needed between now and when we send the kicker email. Here's how the email funnel looks from what I can tell and where we might be able to optimize quickly:

• Emails delivered: 395K in beat 5, the lowest of any of the beats. As a result total sessions on the site are way down in this beat. Optimization: Send kicker to more people (deliverability being the big wildcard here).
• Email CTR: From @stephaniemcv's report these look strong, best of the campaign. Likely has to do with tighter targeting but don't know that there's a lot to do to improve this in the kicker.
• Pledge conversion: Opt-ins vs. Sessions (from GA data) is at 31% for this beat and 39% when just looking at traffic from email. This is way higher than previous beats, largely because we optimized for it whereas previous beats emphasized video views. The big question here is how much more can we get out of this via further optimization. My gut says that 39% isn't bad but we could probably due a bit better given some of the anecdotal feedback we've received.

Based on this, I suggest 1) sending as many emails as we are comfortable with in this final blast and 2) trying some alternative language on the landing page to make it obvious this is a pledge and not just an email signup request.

Thoughts and/or other suggested tweaks to get the pledge #'s up in this final push?

[HPaulJohnson]

@shaghdoosti re: your other data questions:

• Which video had the most shares?

Video 1, with 18K views; video 2 was close behind with 14K; videos 3 & 4 have ~5K each. This is largely a function of overall reach and video views. Videos 1 & 2 benefited from the snippet and much larger email audiences. In terms of share rate (shares/video views), video 4 is far outperforming other beats.

• Which video had the most views?

Videos 1 and 2 (1.7M views and 1.4M views respectively), again greatly influenced by snippet + full list email sends.

Looking at engagement rates (vid views/impressions), videos 3 and 4 far outperformed (20% & 15%). Channel mix is again a big factor as social has a much higher rate.

Looking at the head-to-head comparison of videos 1-3 in the paid media channel, a short cut (:30 secs) of video 2 had our best video complete rate and lowest cost per view.

• Did views of our videos get higher with each beat or did they fluctuate?

No, they decreased as the campaign progressed. I’d argue that views isn’t the best thing to look at here as the channel mix fluctuated so much from beat to beat. Beats 1 & 2 clearly had the advantage of being embedded in the snippet and going to the full email list. Videos 3&4 have had much less reach and, in spite of high engagement rates, aren't producing as many views.

• Did shares of our videos get higher with each beat or did they fluctuate?

No, but for the same reason as above. We reached far fewer people in Beats 3 & 4 so total shares were down after being roughly flat across beats 1 & 2. Share rate remained pretty flat for beats 1-3 before jumping way up in beat 4.

• Which social media post had the most engagement?

The Crypty .gif on FB had the highest engagement rate of the campaign. In general FB far outperforms Twitter for engagement, not to mention reach. Videos on Facebook did particularly well. On Twitter, posts mentioning Apple had the highest engagement along with those that referenced specific media articles about encryption.

• How did social media posts about the campaign (in terms of shares and comments) perform relative to other posts?

Much stronger. Any time we have rich content and concrete things to reference, posts perform better (just like our general comms/PR efforts). Social posts during this campaign performed 50% better in terms of engagement rate than the average campaign (this is a rough benchmark look at Feb/Mar engagement rates vs. previous 6 months average).