Messaging: Cloud Onboarding Emails 3.0

[joethreepwood]

Context

I'm documenting some changes to the Cloud onboarding email flow, so everyone is aware what the flow looks like and can feedback. I'll try to keep this brief, as the underlying logic is complex. Suffice to say, the goal of the onboarding flow is to get users to try new products in our platform, and be successful with them.

It's worth noting that we previously had a stage to check for successful ingestion and used that to trigger other messages, but removed this as it proved unreliable.

Most of the copy for emails is visible here: https://github.com/PostHog/meta/issues/64

Old flow

Here's a description of what the flow used to look like in its most recent iteration

• When a user signs up with a valid email address, we send them the first welcome email after a 1 hour delay.
• We skip over lots of logic which failed to work and which was intended to validate a successful ingestion.
• We wait 1 day and until a weekday, then send them an email upselling them to session replay which directs them to the product page on the site.
• We wait 1 day and until a weekday, then send them an email upselling them to feature flags which directs them to the product page on the site.
• We add the completed_onboarding_email property as true

Proposed flow

• When a user signs up with a valid email address, we send them the first welcome email after a 1 hour delay.
• We then check role_at_organization
  • If founder, and if they have a hubspot_score below 13: We wait 12 hours and send them a personal invite to the startup program from me. We then wait for 12 hours and until a weekday. Preview of that copy: https://github.com/PostHog/meta/issues/121.
  • If engineering, we send them a personal invite from Andy to subscribe to the newsletter. Issue for this: https://github.com/PostHog/meta/issues/122
  • If no: We wait for one day, and until a weekday.
• We send an updated email which explains what an AARRR dashboard is and directs users to the relevant template.
• We check if the user is in the 'subscribed to session replay' segment.
  • If yes: We do nothing.
  • If no: We wait 1 day and until a weekday, then send them an email upselling them to session replay which directs them to the billing page.
• We check if the user is in the 'subscribed to feature flags' segment.
  • If yes: We do nothing.
  • If no: We wait 1 day and until a weekday, then send them an email upselling them to feature flags which directs them to the billing page.
• We add the completed_onboarding_email property as true

Meanwhile, we have two parallel flows for users who signup to flags or replays, or who join an org where it is is already subscribed. These both look like this:

• When a user enters the 'subscribed to replays' segment, we wait until completed_onboarding_email equals true

  • We wait 24 hours and until a weekday, then send the email highlighting relevant tutorials and content
  • We add a property to signal they've completed this flow

• When a user enters the 'subscribed to flags' segment, we wait until completed_onboarding_email equals true

  • We wait 24 hours and until a weekday, then send the email highlighting relevant tutorials and content
  • We add a property to signal they've completed this flow

This updated flow is easier to manage and prioritizes telling users about the additional features of session replay and feature flags. Waiting until completed_onboarding_email = true enables us to prioritize these discovery emails, while removing also pacing the emails so that as soon as a user subscribes to a product they will get emails about getting more value from it within 24 hours.

If a user subscribes to multiple products, they will get two emails -- but only if they've already completed the entire onboarding flow. Arguably not a big issue as they'll be getting those emails at the best possible time for their journey.

Next steps

@raquelmsmith is working with me to create the events needed to create this logic in Customer.io, at which point I'll circulate new copy. Feedback welcome.

[joethreepwood]

Following on from a conversation yesterday about the configuring events with @raquelmsmith , I’ve given the email flows some more thought and realized the initial proposal won't work.

Why: If we only check once that a user is subscribed to a plan, we won't be able to send them the relevant onboarding content for that product if they sign up at a later point.

So, new proposal is added above. It's going to require multiple flows, because we can't have it all in one flow (if a user never subscribes to a product, they'd get stuck in the flow) which is exactly what I was hoping to avoid, but 🤷

[raquelmsmith]

Should we be checking if they have subscribed to a product, or used it at all? And should we be sending them to billing, or to tutorials for how to use?

Generally I feel like people will subscribe if they have sufficient reason, so these emails should probably focus on giving them reason to subscribe (explain that the product exists and the value they can get out of it) rather than just trying to get them to subscribe.

[joethreepwood]

Yeah, I've glibbed over what the actual approach of the emails will be while we nail down the logic.

Right now I'd imagine that the emails for non-subscribed users would be generally along the lines of explaining a valuable use-case (e.g. debug issues faster with session replays) and explaining that there's a free allowance so that subscribing doesn't immediately mean paying, and directing users towards the billing page to subscribe. We can throw in some tutorials to prove the use-case and establish value, but the main CTA would be the subscription.

The emails for users who have subscribed would be a bit deeper on the use-case side and tutorials and the main CTA would push towards the tutorial hub for that product.

Can I assume the logic described above sounds good to you? If that's the case I'll build out the flows and start drafting copy to share here.

[joethreepwood]

I've been over this with @charlescook-ph and have decided on a slightly neater flow that will be more predictable, and hopefully add more value. I've updated the proposal above.

[raquelmsmith]

Yep looks good to me!

[joethreepwood]

OK, proposed copy for the new emails.

Updated AARRR email

Session replay upsell

Feature flag upsell

Session replay welcome

Feature flag upsell

Also, here's the email I've drafted for @andyvan-ph and am using with his approval. I've moved this up in the flow so that it is now used as the second email, rather than the final. I've removed the designed variant for the experiment around startup invite based on results so far. Corrected that in the flow above.

I've also added a replay_onboarding_completed and flag_onboarding_completed attribute to the end of the parallel flows. It'll be handy later.

These are all built now. The remaining issue is around the events for session replay and FF subscriptions, which @raquelmsmith is looking into (thank you!), after which we can roll out pending feedback.

The roll-out will take some time. We'll have to stop the existing funnel, start this one and let users in the existing funnel complete it. That will take at least a week before we can then delete it and have 3.0 fully running.

From here, it'll be a lot easier to build further improvements. Ideas I have for us to test next:

• Further tailoring messages based on role_in_organization
• Lengthening the onboarding campaigns for each product (it's deliberately light now)
• Identifying late-use adoption tactics for power users. E.g. Once a user has created three FFs? Invite them to betas automatically.
• Automate rewards for power users. E.g. When a user is subscribed to everything and has used everything? Send them a laptop sticker.

[andyvan-ph]

I feel like there are too many links in some of these. Maybe it would be better to just link to a tutorial category page rather than each individual guide?

[joethreepwood]

I feel like there are too many links in some of these. Maybe it would be better to just link to a tutorial category page rather than each individual guide?

We already do that in the first welcome email for all users, directing users to the docs and tutorials.

Idea here is really to ram home that there are lots of things they can do and make sure they're aware of the various use cases. We don't actually expect someone to immediately see it and go 'ah, yes, I should do a canary release' right away - but we want them to know it's possible.

(Not to dismiss the feedback, but it was something I thought about)

[charlescook-ph]

Honestly only text feedback I have is for Andy's email - rather than saying 'technical people', how about 'people like you'? (I get you're trying to not say engineer twice, but 'technical people' sounds like something very non-technical people would say).

[ivanagas]

• For "Feature flag upsell," we cannot target by user behaviour (other than we use properties). We shouldn't recommend behavioural flags.

[joethreepwood]

Honestly only text feedback I have is for Andy's email - rather than saying 'technical people', how about 'people like you'? (I get you're trying to not say engineer twice, but 'technical people' sounds like something very non-technical people would say).

Sure, will fix. FYI to @andyvan-ph though that this language was cribbed from https://newsletter.posthog.com/about so he may want to change it there too.

• For "Feature flag upsell," we cannot target by user behaviour (other than we use properties). We shouldn't recommend behavioural flags.

Good catch 👌

Will start this launching once the event issue mentioned above is sorted.

[joethreepwood]

This is now launched and, so far, seems to be doing pretty well! Couple of nice follow-ups to the personal emails, and a smooth launch. 2.0 is slowly closing for users in that flow already. Will re-open if needed, but for now...