PostHog / meta

This is a place to discuss non-product issues in public.
MIT License
18 stars 4 forks source link

Onboarding 3.3: Tweaks & Experiments #140

Closed joethreepwood closed 1 year ago

joethreepwood commented 1 year ago

Documenting some small tweaks and experiments within the onboarding email flow.

Email to founders

We've previously disabled the email which is sent to founders, inviting them to join the startup program. This may be temporary while we investigate potential cannibalisation.

Until we complete that investigation, I've now replaced that email with an invitation to subscribe to the Substack newsletter. I've kept it addressed from me, and generally tweaked it.

We know that this email converts well for Engineers, at a rate of almost 6%, so it will be interesting to compare how it works for founders.

Screenshot 2023-10-13 at 11 24 36 Screenshot 2023-10-13 at 11 22 08

Adding signatures

We previously had several emails from individuals -- mainly @andyvan-ph and me. I've now tweaked these to use the correct email signatures for us, as it's been my experience for the startup email that replies are regular and it's be incongruous to have discrepancy here.

Screenshot 2023-10-13 at 11 23 44 Screenshot 2023-10-13 at 11 27 00

Changing subject lines

I'd previously been trialling three different subject lines for the other welcome emails which are sent to Product, Marketing, and Sales users.

Short version? The variant for sales teams - A helpful tip for PostHog - performed best. It was almost +20% better on the open rate, and 2x better in terms of the click rate. We may want to adjust the copy too, but for now I've rolled this subject line out to the other emails.

Screenshot 2023-10-13 at 11 38 58

Experiment: Personal invite to try replays

The session replay doesn't do well, with a 1.1% CTR over the last 12 weeks. Maybe a personal approach would work better? I'm competing it against this copy in an A/B test.

Screenshot 2023-10-13 at 12 10 20

Experiment: Email variant for flags

The feature flag variant doesn't work well either, with a 0.8% CTR over the last 12 weeks. Maybe we aren't being clear about the potential use-cases? I'm competing it against this copy in an A/B test.

Screenshot 2023-10-13 at 12 21 04