Messaging: Surveys upsell email campaign

[joethreepwood]

Messaging

Context

We previously did a session replay upsell campaign. 3 emails, over 10 days.

We learned that a particular, personal style works best and bought around 20 new users into trying session replay for the first time. Now, I want to do a second test targeting getting users to sign up to a survey.

Targeting this because it's a feature which isn't widely adopted or publicised yet, would have wide relevance and isn't massively technical for non-engineer users. Creating a survey will also require users to opt-in to the early access, which I think is a very important thing to make visible to them. Users who see this screen will be more likely to join other betas.

Goal

Get users to create their first survey within two weeks of receiving an email.

Format

Email, customer.io. Broadcast x3.

Audience / Targeting

As before, I'll target paying users who have no previously created a survey and who have logged in at least once in the last 30 days (e.g. active users).

The cohort is around 1303 people.

Resources, timeline, responsibilities

@joethreepwood leading. It'll run as three broadcasts, over a period of 10-14 days to a targeted group.

Requested input

Any feedback welcome.

Previews

Drafts https://docs.google.com/document/d/1VbWAVQioxmTXahtURISSY5IDwO3XYsXMN0hLrYuS9pk/edit?usp=sharing

[annikaschmid]

@joethreepwood Surveys is available for everyone (in beta) now, so no need anymore for early access.

[joethreepwood]

You just had to throw a spanner into the works 😂

I may keep this focused on Surveys anyway, as the other EA items are either a bit less actionable or more complex atm, so I'm not sure an email campaign at this point is the right approach.

[joethreepwood]

OK, I'm going to keep this focused on surveys and I'm going to run two variations. Both will be simple, short, personally styled and directly from me. But one will have gifs to give a visual element.

@annikaschmid / @andyvan-ph would love to get your quick eyes on the copy, please!

https://docs.google.com/document/d/1VbWAVQioxmTXahtURISSY5IDwO3XYsXMN0hLrYuS9pk/edit?usp=sharing

[joethreepwood]

There's been no feedback on these, so I've set up the broadcasts as per the drafts and am planning to launch them next week.

[ivanagas]

We have two survey related tutorials now if you want to use either of them:

• https://posthog.com/tutorials/feedback-interviews-site-apps
• https://posthog.com/tutorials/survey

Otherwise looks good

[joethreepwood]

This is now live and scheduled to send over the next 10 days, with a few days between each message..

[jamesefhawkins]

@liyiy are surveys ready for this? i'm not quite sure how we feel about quality at moment...

[liyiy]

@liyiy are surveys ready for this? i'm not quite sure how we feel about quality at moment...

My assumption was that this was more of a beta test upsell but if not then yes I think waiting maybe for another two weeks or so while I get in some more features should help for a more impressive general release upsell

However I don't think this hurts either as the base offer of surveys right now seems to fulfill an okay amount of user's requests and it'd be really good to get more exposure so we can receive more feedback on it too! 😄

[joethreepwood]

So, we ran this upsell but ran into two main hiccups. The first is that, due to confusion with configuring the emails as multiple broadcasts, the first email in the sequence wasn't sent. The second is that we had 77 of the emails fail to deliver.

The first issue is my fault. The second is just bad data, nothing to be done about it.

Despite this, we had a good response. Overall stats:

Open rate: 52% (varies less than 1% depending on email) CTR: 5-8% (depending on email) Unsubscribes: 0.1% (1 user) Conversion rate for making a survey: 0.6% (6 users)

Most crucially, we got actual user feedback!

• One user reached out to me directly to express interest in NPS surveys (tutorial incoming: https://github.com/PostHog/posthog.com/pull/650 )
• One user reached out to Li to give feedback on surveys
• One user reached out to ask why behavioural cohorts can't be used
• One user reached out to comment on the emails themselves and ask to be unsubscribed

The test revealed that the variants without a gif performed slightly (1 conversion) better than those with a gif, and had a higher click through rate too.

All in all, seems to validate doing targeted emails like this for new features and keeping to a format (as with the previous test) which is direct, personal and doesn't use gifs or images.