search

PostHog / meta

This is a place to discuss non-product issues in public.
MIT License
18 stars 4 forks source link

RFC: Support Hero Bot #59

Closed joethreepwood closed 1 year ago

joethreepwood commented 2 years ago

Normally, I'd just look at doing a PR for this idea -- however the solution wouldn't be on GitHub and I don't have access to/full visibility of all systems involved (Zendesk, Halp).

Context & Problem

Our support scores are currently an area of weakness in our G2 reviews, indicating that users aren't very happy with the level of support they receive.

After looking through much of our support documentation and what's available in Zendesk, Slack, etc. I have a few hypotheses and areas where we can improve -- though it's worth stating that this is mainly an issue for users in the Community Support channel.

As indicated by the G2 response, poor support experiences can create negative word of mouth.

  1. It's not clear to users who/what Support Hero is. Users may not be aware of what Support hero is as a concept, how it works or whether they are receiving support from a community or staff member. Much of this information is listed in the Support Hero docs, but this content isn't signposted to users, is very lengthy, and isn't wholly accurate (3). Users may not know what timezone support hero is in, or how this changes.

  2. We don't set appropriate expectations. Users likely have different expectations for response times based on the channel they send requests to. Additionally, while many users are only sold community support, they actually still get official support from a developer -- again, faulty expectations. It is not clear to users how issues are prioritized or when they get a response.

  3. We don't seem to use all processes described in the Support Hero docs. The support hero docs describe a few systems which don't seem to be in regular use, such as adding ticketing emojis to move Slack issues to Zendesk (I've never seen one) and directing users to self-serve tools, such as Squeak. Even better, we want users to help users.

What to do about it

I'm proposing that we introduce a Slack automation that attempts to solve some of these issues. Basically, whenever a message is posted to the Community Slack channel, the Support Hero bot responds with a basic message to give information, set expectations and direct users:

It looks like you're asking the community for help. Our Support Hero this week is <name>, who is based in <Timezone>. We recommend checking [our Support site](https://posthog.com/questions) to see if your question has already been answered - but, if not, we'll respond [as quickly as we can.](https://posthog.com/handbook/engineering/support-hero#prioritizing-requests)

andyvan-ph commented 2 years ago

Sounds like a good idea to me. Would this be a DM to the user, or a reply in the thread?

joethreepwood commented 2 years ago

I'd see it as (because it's probably the easiest approach) an automated reply in thread to every post in the community support channel only. Users with premium support can then continue to get 1:1 support via their dedicated channels.

I should add: I'm happy to have a go at implementing this (with a stretch goal of creating an event in PostHog somehow), though I expect I'll have a few questions about how best to pull in name and timezone fields. The lower-hanging fruit I can do more quickly would be a more general message.

andyvan-ph commented 2 years ago

Minor issue which may impact that. Sometimes people aren't great at Slack discipline and post a query over several messages, rather than one. It might result in a bit of noise, but probably not a big enough problem to change things.

charlescook-ph commented 2 years ago

I assume any solution we have here could include some logic where they only receive this message if they haven't posted a support message in (say) the last month?

simfish85 commented 2 years ago

Would it be easier (and less noisy) to have an automated way of updating the channel topic/description to handle who the support hero is, what timezone they are and the expectations? It feels a bit noisy to me to have a bot responding to every question which comes in? (Also could make Zendesk noisy too)

joethreepwood commented 2 years ago

Ah, the Zendesk point is a really good one. I hadn't considered that.

My worry with updating the channel is that that post would then just get lost in the noise. The context for this channel means nobody is going to be scrolling back looking for a pinned post or anything, so I don't think the (great) approach you have for dedicated channels would have much impact here.

simfish85 commented 2 years ago

When you set a topic/description for a channel it's always at the top of the page though? I've highlighted it in this image. For dedicated customer channels we usually have the support here tagged there so (for me) it's visible.

Screenshot 2022-08-09 at 16 01 14

I am also aware that people might not check it though...

joethreepwood commented 2 years ago

So, I've spent some time today looking into this.

Option 1: Zapier

It should be possible to get a Zapier app to offer basic information to users in thread replies. However, it would come with two limitations.

I've tried setting up a Zap to get this working, but it seems to fail for reasons I can't quite discern. Possible I may be able to get this going with some support, but pausing on it for now.

Option Two: Slack Pro

We could create a Slack app using the workflow builder. However that has two issues:

Option III: Make a Slack API bot

It should be possible to create a Slack bot using the API to solve our needs entirely. We could apply logic and everything.

Downsides:

The only other options I can see would be the changes @simfish85 described above, but I'm not convinced they'd have any impact.

What I'd propose is that we go ahead and put some engineering time into making a Slack API bot. I'll put a feature request in for now and try to come back to doing it myself if there's no traction.

simfish85 commented 2 years ago

Agree Option 3 is likely the best way to go. For 1, @camerondeleone and I have been having adventures in Zapier recently so would be happy to help you out/take a look. Could you not also use Zapier to query PagerDuty for the on-call person?

joethreepwood commented 2 years ago

Agree Option 3 is likely the best way to go. For 1, @camerondeleone and I have been having adventures in Zapier recently so would be happy to help you out/take a look. Could you not also use Zapier to query PagerDuty for the on-call person?

ooh, there's an idea RE pagerduty. I'll have a look at that.

I've built a Zap already which is supposed to reply in-thread by comparing Timestamp values (apparently it's the only way to get thread messages, rather than channel messages) but it isn't currently working. Welcome to take a look at it!