Messaging: Session replay upsell campaign

[joethreepwood]

Context

@simfish85 and I want to run a test of upselling a targeted portion of users who are engaged with PostHog but not using session replays, to use session replays. It'll be a short flow, targeting a custom segment that we've specified together.

Goal

Getting these users to enable, or engage with session replay as a feature.

We'll track open and click-through rates as normal, but will export the segment back out of Customer.io and into Pocus to measure activation rates and any revenue impact.

Format

Email, via Customer.io.

Audience / Targeting

We've begun with a segment of 915 users identified by CS who are paying users that have not activated session replay.

We're further narrowing this segment to focus on users who are currently actively engaged. Users must have completed the onboarding flow in the last 30 days, and have logged in in the last 14 days.

Resources, timeline, responsibilities

• [x] Simon to create the initial list
• [x] Joe to draft the initial content
• [x] Joe to build the emails

Requested input

Ideally we would get some dev assistance with this PR - https://github.com/PostHog/posthog/issues/15608 - so that we can most effectively link users to a very simple, direct modal to enable recordings. So, that's one ask.

Failing that, we'd just have to link to the settings page and trust that users can find the option on that very long page.

Previews

Drafts.

Coming soon.

[joethreepwood]

Good news/Bad news: We aren't immediately able to filter this campaign to focus on users who have completed the onboarding flow within the last 30 days because Customer.io doesn't support this. The list is small enough, however, that it shouldn't be a blocker to progress and, for future tests in this direction I've updated the onboarding flow to add a new onboarding_completed_once: true attribute whenever someone exits it.

[joethreepwood]

Additionally, going to use app.posthog.com/replays as a CTA, instead of settings. That page has a call-out for the modal which means it can function better as a place to send people.

[joethreepwood]

@simfish85

There are two options to consider here, in terms of layout and approach across the three emails. The first is more of a marketing email, with a bit more design. The second is presented to look much more like a direct email from you.

Do you have a preference? Or we could A/B test them across the campaign.

[simfish85]

I think this would be great as an A/B test @joethreepwood

[joethreepwood]

Quick screenshots below for approval. All emails will look like they come from @simfish85 and replies will go to him. I've set a goal on the campaign: Generate a 'recording analyzed' event within 2 weeks of open.

One thing to note @simfish85 is that, of the list of 900ish users you shared, less than 300 of them match the criteria of also having logged in over the last 14 days. So each variation for each message will only get around 115 users in it, vs. the advised 500 minimum.

We can push forth as-is, but we may want to consider dropping the 14 day criteria to get a better result and a more statistically relevant read.

@simfish85 let me know what you think and, once I had a thumbs up, I'll set these to send over a 5-day period.

Email one:

Email two

Email three

[joethreepwood]

(there's no automation set up for the sticker promise btw, so we'd have to handle manually this time)

[andyvan-ph]

One email is fair enough, two is on the line, three feels excessive, especially over a five-day period. I get people can opt out whenever, but I feel strongly we shouldn't reach a point where people feel the need to do so. Most people will just block us rather than opt out.

[simfish85]

Can we personalise the 'Hey there' to 'Hey Joe' ?

[joethreepwood]

One email is fair enough, two is on the line, three feels excessive, especially over a five-day period. I get people can opt out whenever, but I feel strongly we shouldn't reach a point where people feel the need to do so. Most people will just block us rather than opt out.

Agree with the time period on reflection. @simfish85 How would you feel if we scheduled this over a 10-day period instead? Like a Monday/Friday/Friday cadence.

Can we personalise the 'Hey there' to 'Hey Joe' ?

Unfortunately, it doesn't seem like we pass name information to Customer.io, so we can't do this.

[corywatilo]

Have you tried PostHog's session replays yet?

Would probably change this since, as an analytics company, we should know if they have/haven't tried it. (In fact we do, but they may wonder.)

Instead something like, "I noticed you haven't tried session replay yet."

[simfish85]

@joethreepwood fine with the 10-day cadence! Let's get it going

[joethreepwood]

Scheduled:

Email 1: Monday 5th June Email 2: Friday 9th June Email 3: Friday 16th June

[joethreepwood]

The third email isn't sent yet, but a quick look at the results for 1 and 2 suggests that the more personal, non-designed version performs slightly better. Will loop back in a week with full results.

[joethreepwood]

Results are in:

• Average 55% open rate across all emails vs 3 total unsubscribes
• 7% conversion per email: 41/200 people tried replays for the first time

• 8 people clicked a link to book a call

  The personal style generated almost twice as many conversions, except for the last email in the sequence.