NoK: Medallia user feedback analysis since June 10

[wesrowe]

Description

User story

As a Cartography team member, I want to request qualitative feedback frequently after launch of NoK/EC, so that we can identify issues that Veterans experience.

Notes

• feature launched 4/8
• example request from 4/11 thru 4/19
• NOTE: phone number was fixed (replaced with "call a facility") around 5/8
• Sharepoint folder to save data – see "NoK user feedback - aggregated.xslx"

Possible tasks:

• request data

Acceptance criteria

• Relevant feedback noted in comments below and escalated to team for remedy
• Data saved to team Sharepoint site alongside previous analysis

[wesrowe]

Requested data 9/5 – https://github.com/department-of-veterans-affairs/va.gov-team/issues/92289

update: uploaded pulled data to Sharepoint folder and pasted it into the aggregated feedback Excel file there.

[wesrowe]

@jonathanjnelson @fmccaf1 @katbrinkley, FYSA I pulled this feedback and am reading through it. Insights:

• People seem to be getting the help they need now that we posted the HEC's phone number. Someone even thanked their HEC rep by name (kudos to Thomas!)
• There are at least two people giving feedback about medical supply reordering, eg that the form didn't work, or can't find tracking info for a shipment. Makes me wonder how they ended up in Profile before giving that feedback? (using VFS/button survey)
• Honorable mention for a multi-faceted item that includes both edit-Nok/EC and wanting to see priority group
• Multiple Veterans assume that they can't edit contacts online for security reasons; a few of these seem to think that their browser/laptop is what's preventing it
• At least one person feels their privacy is violated by having to discuss personal contacts with a provider (by calling)
• Multiple complaints that changes done in-person at VA don't show later on VA.gov (3 around July 17)(2+ more in Aug)(8/7 comment specifies multiple systems that were updated already: DEERS, VA, DMDC)(2 on 8/16)(8/18 again)(9/5 "called" but no update)
• Some people don't even see that they can call HEC, so don't know how to change contacts at all (presumably they're not opening our expanding info component) – MOVE HEC number to plain text (not hidden)?
• 7/29 complaint the contacts page doesn't fit smartphone screen... investigate? on samsung galaxy s20, but "usable screen resolution" is just 320px wide. maybe zoom user?
• 8/14 "phone number doesn't work" – this is after we added the HEC number in early Aug, ~8/5.
• 8/12 & 8/18 – there are "instructions online" that NoK/EC can be edited online. (one person mentions google) – TRY GOOGLE TO REPRODUCE
  • 8/23: "sent to site to update health contacts"
• Common use case: Veteran moves, their NoK/EC moves with them (e.g. spouse); but they can only update their own address online

Other observations

• 2 people are mad that the name entry field (probably preferred name) doesn't use the capitalization they entered – e.g., MacD becomes Macd; Smith-Anderson becomes Smith-anderson.
• a word cloud of people's reactions to not being able to edit would include "ridiculous" "stupid" and "STUPID"

Impact?

• 132 / 350 responses include the word "contact" in the task description field – about 2 per day, and likely many of those who left that field blank were also trying to change contacts. Each of these complaints represents at least 10 users who didn't bother to complain. So a conservative estimate of how many Veterans are surprised/upset by no edit feature is 25/day.
• 1500 Veterans visit the /contacts/ page per day (average for Aug). Do we really believe that only 25 of them found that their contacts were wrong? That doesn't seem plausible. Highly likely that people are under-reporting complaints due to low expectations for VA.gov.
• per GA, 25% of users opened the "how to change" component. this suggests a high proportion of users see something that needs correction.