search

PostHog / meta

This is a place to discuss non-product issues in public.
MIT License
18 stars 4 forks source link

Messaging: Outage apology - Feb 2024 #181

Closed joethreepwood closed 8 months ago

joethreepwood commented 9 months ago

Messaging

We had an outage which impacted all customers. It lasted 15 minutes and resulted in lost data for users. We should say sorry.

Two ways to do this:

  1. We update the /service_message page and send a notification in the app, directing users to it
  2. We email all users directly

In either case we need to explain:

  1. What happened
  2. The impact of it
  3. How we'll stop it happening again
  4. ~It was a DNS issue~
  5. That we take responsibility
  6. What happens next (nothing in this case)

Let's assume for now that we go the email route.

Screenshot 2024-02-15 at 14 37 10
Twixes commented 9 months ago

Public post-mortems are a good way to win confidence that we're serious about issues like these

joethreepwood commented 9 months ago

We're already discussing if we can make the post mortem public :)

simfish85 commented 9 months ago

Looks good - should it come from customers@posthog.com in case anyone replies? hey@ is technically unmonitored

corywatilo commented 9 months ago

US date/time formats! ;)