[Data] 1010 Voice of Customer Reporting & Cadence

[luis-simauchi]

Lois Lewis wants to stand up a regular cadence of reporting on the health of the CG and other forms, gathering the customer's sentiment on the digital experience. She wants to model the report & form metrics to be gathered after what The Benefits Team is doing.

[ ] Success: Work with Lois to compile this information for OCTO purposes and share it with the team for learning.

[luis-simauchi]

We are planning on discussing the details this afternoon during the weekly design stand up

[luis-simauchi]

Lois wants 2 things: -- To mirror the metrics measured against for the forms generally HEC stakeholders -- To tie metrics measured to story driven research that drives changes

staring this with the call center data & medallia feedback data

[luis-simauchi]

Proper work session with Lois set for Wed @ 10:30

[luis-simauchi]

Analytics chat NOTES from yesterday with Lois: Voc = Voice of the Customer, effort from CIO office Team to gather data that exists; processing and presenting to CIO, Lois involved with that effort - separate from this team What benefits team is doing; way of working + align with what they are doing Monthly pull basis Medalia More actionable insights Tell the story with links back to data CSAT score positive or negative — what happened? What we know? Starting pinning to certain changes made on form Concerns with data sources populating correctly Need to do a clean up = landscape effort Leaning more into GA4 and DataDog Start reporting richer metrics ACTION ITEM: Team next steps meeting, PM + PO focused with Luis and Lois

[luis-simauchi]

lois would like our Medallia comment data studies / process to be more visible

[luis-simauchi]

here are the notes/action items: Landscape (categorize feedback) VOC by End of Sept (tentative, up for discussion) Pull Aug 24 within first week of Sept Maintain on a monthly basis Think about prioritization framework and metrics model in Benefits deck (shared below) Luis will analyze the data sets available & clean / transform for viewing

[emily-brown34]

let's create a ticket for EZR @alexseelig and another for EZ @hdjustice also to make sure we capture all VoC data, then moving forward we will create a monthly ticket for all three forms to review and package the feedback

[luis-simauchi]

Call Center Data recap: -- its all Hotline tier 2 -- case type is not a useful field, -- not a lot we can glean for the call center data, most of it is centered around disability claims with a sprinkle of caregiver mentions

[luis-simauchi]

Lois' ideation framework: -- current CSAT indicators make it challenging to respond to specific pages / products -- revisit how we do metric collection

question from the above: -- Can we get help desk data from ??? -- Do we know the response rate of the our surveys (yes but its not a direct measure, we can calculate it based on application starts)

[luis-simauchi]

my response: -- Medallia does give us deeper insight into what pages might be problem spots for users -- one way to drill down, imperfect, is to filter down on page and see the distro of rating by pages, see where the worst rating lay, then check the comments for unifying theme of issue with that page.

[luis-simauchi]

another way to spot problems: -- did you complete you task today question: yes / no -- best indicator for "is there a problem on page" -- can filter down to page afterwards or a step further by sifting through comment data for keywords that identify the problem

[emily-brown34]

will review with tag up with lois and design team later today

[luis-simauchi]

https://app.mural.co/t/departmentofveteransaffairs9999/m/departmentofveteransaffairs9999/1707845423191/aae3ef198736363ce307f432412eb02fc09b932e?wid=0-1707849890167&sender=u41cb6f1399f421811cfb6106

CSAT homework mural where take away

[luis-simauchi]

Share the excel spreadsheets that I have been working on with the design team, need to strip PII from them first

[luis-simauchi]

set up a metrics readout similar to the one we do for EZR, across all forms. Then a QoQ % & MOM % measure for each reporting period

[luis-simauchi]

EZ comment data shared with the design team