search

PostHog / meta

This is a place to discuss non-product issues in public.
MIT License
18 stars 4 forks source link

Q4: SLA Achievement Improvement Ideas #246

Open benHPostHog opened 1 month ago

benHPostHog commented 1 month ago

SLA Achievement Improvement Ideas

Below are a list of ideas that could improve our SLA achievement rate whilst keeping CSAT levels the same (or even improve them!) The purpose of this issue is for anyone who would like to have an input, can. Please say what you like, what you don't like and suggest any other ideas you have. Once everyone has had their say, this list will be defined and made part of the Q4 goals. The list is in no particular order.

Ideas

Changes Implemented:

joethreepwood commented 1 month ago

Looks good as a starting point!

  • Create more macros / predefined responses / snippets

Would be good to get some more input from engineers about what's useful here, plus making sure they actually use them. Basically good if we can flesh this out a little when we do it.

  • Should low-priority tickets have a defined response target?

No feeling here. Would be interested to know what @abigailbramble and @slshults think if they have strong thoughts.

Provide an incentive for CSAT survey response, such as purchasing a meal for a hedgehog per response.

<3

Are all tickets are included in the breaches report?

Not sure what you mean here?

I think there's also a point we could do around combing tickets for tutorial ideas. If there are certain things which keep coming up or getting asked then we should put that into action (either directly or via the marketing team) as a tutorial. Stop that question getting asked so much in the future.

In the past I've done this as a weekly task where I just looked at one-touch tickets and threw them into #content-ideas channel. It's a quick approach, I just fell behind on it. Would love if we could roll that into this task for you both!

jamesefhawkins commented 1 month ago

i'd add:

benHPostHog commented 1 month ago

Something I have noticed today after thinking about the above ideas, and I could be wrong, is most of the tickets that currently breach SLA are those we do not know the answer to as a team. I think we need to be more proactive about what gets escalated before breaching SLA and set guidelines to follow for this. Seems obvious, but I didn't write it down so...

slshults commented 1 month ago
  • Create more macros / predefined responses / snippets

Surely! Have at it. When you find something that you're repeating a lot, feel free to create a macro for it. (You can mention it in this thread to let others know about new macros.) I was thinking the same during my first month, but it wasn't long before I realized that don't send the exact same response, or partial response, very often. And when I do, it tends to be single sentence that would just become clutter in ZD, so I use a Raycast macro instead (.e.g. "You can just copy/paste the URL from your browser's address bar while viewing or editing the insight." )

  • Should low-priority tickets have a defined response target? Replying to a low-priority ticket after weeks can be seen as insulting

Currently low priority tickets are silently auto-closed after 30 days without activity, and the SLA for them is set for 60 days to avoid having them impact SLAs. It's hard to predict how different users might react to getting a reply weeks later, but we're not promising support to users who are taking the free route, so hopefully some will also be pleasantly surprised. I'm comfortable with our status quo on low priority tickets (at least unless/until we're staffed up to a level where we can actually make promises regarding response times for non-paying users.)

  • Reduce the number of notifications for SLA breaches in Slack. Right now, they are too easy to ignore.

We've been seeing a lot more of these than was the norm before we fell behind during onboarding week, and we're not fully caught up yet. Before onboarding week, we were only seeing a few of these per day (other than product analytics.) Currently there are three alerts prior to each breach. What do you think of keeping an eye on this for another 3 or 4 weeks and then see if it still seems like more noise than signal then?

  • Make the request for feedback after a ticket is closed more prominent
  • Provide an incentive for CSAT survey response, such as purchasing a meal for a hedgehog per response.

I also love this!

  • Big effort on cleaning up Zendesk. Right now there are so many tickets which look like they should be closed out, less noise makes issues more obvious

In which view(s) are you seeing that? Any high and normal priority tickets in views that we work in are tickets that need to be addressed, so the high and normal priority sections of our views should currently be all signal, and no noise. (We do have a lot of old, open tickets many that have been open since last year. The SLAs were only set in ZenDesk in April of '24, and the SLAs are ignoring tickets created before then. There is certainly room for a big clean up, but, it doesn't feel urgent to me since it would take time from answering tickets, and doesn't seem to be slowing us down afaik.)

  • Are all tickets are included in the breaches report? If so, create a view showing tickets breached and close to breach.

Hmm, what's the "breaches report"? Current views have an SLA column which shows orange flags with a time for tickets nearing breach, and red flags for tickets that have already breached. (If you're working from a view that does not have the SLA column, definitely add that column.) I'd be concerned that a breach-specific view would cause us to omit tickets that should be flagged as nearing breach or breached, but aren't (ZenDesk is imperfect when it comes to those flags, so I'd have trouble trusting views based on SLA flags only.)

  • A Brown Bag for complex tickets

I like the sound of that. What would be a good example of the complexity that you have in mind, and what might we hope to improve/solve in a Brown Bag focused on complex tickets?

  • some way (AI summarising Zendesk?) of communicating product feedback from the issues so we get fewer tickets in absolute numbers, by making it easier for us to know what is bothering "lots" of users so we can prioritize those features/bugs/UX work more easily

I like that concept a lot, @jamesefhawkins , but I am concerned about letting an AI crawl through our tickets, which contain PII of our customers, and sometimes PII of our customers' customers. (In a perfect world I'd have also brought a suggestion for allowing AI to crawl our tickets without scooping up PII in the process, but I'm drawing a blank atm.)

@benHPostHog , regarding "most of the tickets that currently breach SLA are those we do not know the answer to as a team.":

We shouldn't be letting tickets sit without a reply if we don't know the answer. We should be finding the answer and/or escalating those tickets to engineers who would be able to answer them. We should be working on tickets in the order they were opened, within the brackets of High, Normal, and Low priority (aka, working on the tickets that have been in breach the longest, or are nearest breaching.) So, if I'm understanding correctly, it sounds like the solution here is that we shouldn't be cherry-picking tickets, we should be working tickets in the order they were opened, as noted at the bottom of this section of the support hero page. (Let me know if I'm just confused though! 😊)

benHPostHog commented 1 month ago

Another tiny thing, I am very guilty of replying to a ticket quickly if I know the answer, and forgetting to assign myself. I am used to previous ticketing systems assigning tickets to me if I make a public reply. So far I don't think Zendesk has been doing this.

joethreepwood commented 1 month ago

You can mention it in this thread to let others know

I have no idea what the solution is here (and this isn't a goal for this project) but I would love to have a more elegant solution than this. Wonder if we should just be feeding small changes straight into the main channel, instead of thread. 🤷

darkopia commented 1 month ago

You can mention it in this thread to let others know

I have no idea what the solution is here (and this isn't a goal for this project) but I would love to have a more elegant solution than this. Wonder if we should just be feeding small changes straight into the main channel, instead of thread. 🤷

What about this? https://github.com/PostHog/posthog-zendesk/blob/main/CHANGELOG.md But then its not really as visible which is the goal.

Perhaps even just a changelog issue we can subscribe to, just like this thread?

joethreepwood commented 1 month ago

Reduce the number of notifications for SLA breaches in Slack. Right now, they are too easy to ignore. What do you think of keeping an eye on this for another 3 or 4 weeks and then see if it still seems like more noise than signal then?

On this, I think we should try to make a decision now. If we wait 3-4 weeks, we basically lose a large part of the quarter without making a decision in the hope that the problem will go away, which isn't going to solve anything because:

  1. This situation will definitely repeat because it's in the nature of CS tickets to grow no matter what
  2. The fact that these are easy to ignore is a problem we need to solve now and which will continue even if the volume drops

I don't have a strong feeling on what the solution here is. Some ideas I'd suggest we consider are:

  1. Increase the signal-noise ratio by removing the second alert. That way we basically get a gentle nudge when a problem is looming, and then a final warning when it's on our doorstep. The middle alert doesn't feel like it adds much here.
  2. Making it so that the SLA alerts tag whoever is assigned using their Slack handle, so notifications go straight to the individual to increase accountability.
slshults commented 1 month ago

Another tiny thing, I am very guilty of replying to a ticket quickly if I know the answer, and forgetting to assign myself. I am used to previous ticketing systems assigning tickets to me if I make a public reply. So far I don't think Zendesk has been doing this.

Yeah, different teams use the assign feature differently, so we don't have ZD assign new tickets to people ('trust over process'), or when replying. (It will assign a ticket to you if you mark it Solved though.)

darkopia commented 1 month ago

Another tiny thing, I am very guilty of replying to a ticket quickly if I know the answer, and forgetting to assign myself. I am used to previous ticketing systems assigning tickets to me if I make a public reply. So far I don't think Zendesk has been doing this.

Yeah, different teams use the assign feature differently, so we don't have ZD assign new tickets to people ('trust over process'), or when replying. (It will assign a ticket to you if you mark it Solved though.)

@benHPostHog Sorry, I totally forgot to reply on this saying I am guilty, too! Old habits die hard. After the 3rd time someone assigned a ticket back to me when I had forgotten, I made a trigger for myself that just removed the problem. In an ideal world, we should take the ticket first so no one else starts on it, but if we forget, there is no harm in a trigger catching it. Fewer clicks too :D Could easily add you on https://posthoghelp.zendesk.com/admin/objects-rules/rules/triggers/29719074522523

darkopia commented 1 month ago

We made a minor change, and we can monitor its impact: CSAT emails are no longer sent on the weekend, when people are less likely to be working.