search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

Analyze user feedback from Medallia surveys for past 6 months #70048

Closed wesrowe closed 11 months ago

wesrowe commented 12 months ago

Description

User story

As a Cartographers team member, I want to analyze the qualitative survey responses we received from the past 6 months, so that we can learn from it.

Notes

Acceptance criteria

Tasks

sterkenburgsara commented 11 months ago

Documentation / lightweight analysis in GitHub here.

sterkenburgsara commented 11 months ago

@wesrowe I'm not sure there are clear follow-on tickets to create here based on lightweight analysis. We could discuss whether the overall rating in Medallia could inform a similar KPI for the transition (e.g. do we think that the same ratio of users having a good v. bad experience will stand? Deviate by some %?).

wesrowe commented 11 months ago

@sterkenburgsara,

re no obvious follow-on tickets: no problem. sounds like no users had specific bugs with the landing page.

re "overall rating" – what I've heard is that MHV stakeholders are proud of a satisfaction score (in the 70s?), and we will want to measure the equivalent and track to it. There might be a clue in the VAgov dashboard in Domo. The very first card in the "All VA.gov dashboard" is Avg satisfaction score. It has been in the low 50s % for the last year or so, actually used to be a bit higher a couple years ago.

Your analysis was really complete, I thought! Thanks for the thorough job. The one thing missing is the average scores for the two surveys (somewhere between 1 and 5). Is that easy to grab?

Note for later: we'll need to figure out how the % c-sat score is calculated – it might be similar to an element of your analysis, where you counted 4s and 5s as good/high scores.

EDIT: I made a few small edits to the report to add detail about the two surveys, e.g. one is intercept, the other is feedback button.

wesrowe commented 11 months ago

this can close

sterkenburgsara commented 11 months ago

@wesrowe interesting re: score in the high-70s, I will peek at that, but it is not what we are seeing for these surveys. Tests are laid out somewhat differently, but yes - I was trying to bubble up a CSAT score, basically. I went back and looked at the data to add in a comparison metric for the second survey.

Thanks for your edits!

sterkenburgsara commented 10 months ago

@wesrowe we are missing the last month-ish of data from Medallia. Happy to recalculate if we can get that & see if it nudges the CSAT closer to #'s you're seeing in Domo.