Build Proof of Concept for Product Success Tool MVP in Domo

[nedierecel]

User Story or Problem Statement

As an Education Product Owner, I want to test the Product Success Tool MVP within Domo so I can make decisions and iterate on my products.

Goal

• Recreate custom Product Success Tool MVP within Domo to see how close we can make it to the custom approach

Objectives or Key Results this is meant to further

O2: Customers use qualitative and quantitative user data to build and iterate on VFSs.

Acceptance Criteria

• [x] Use Google Analytics native connector to build non-BigQuery dependent charts within Domo
• [x] Customize dashboards to fit VA.gov style guidelines
• [x] Calculate error rates using GA data
• [x] Align Foresee data with Education product area
• [ ] Communicate with Education stakeholders about using the POC
• [ ] Stretch: Customize dashboards to fit Grey Sky/customer focused designs
• [ ] Stretch: Map which features will need more work beyond a sprint (i.e. % Difference from Target over time)

Acceptance Criteria for Data to be Included

• [ ] Available Google Analytics data (using the native Google Analytics connection)
• [ ] Available historical ForeSee data (using the pulled CSV file)

How to configure this issue

• [ ] Attached to a Milestone (when will this be completed?)
• [ ] Attached to an Epic (what body of work is this a part of?)
• [ ] Labeled with Team (product support, analytics-insights, operations, triage, tools-improvements)
• [ ] Labeled with Practice Area (backend, frontend, devops, design, research, product, ia, qa, analytics, call center, research, accessibility, content)
• [ ] Labeled with Type (bug, request, discovery, documentation, etc.)

[amycesal]

SO clicking a form in the "all values" section selects it BUT clicking a form in the legend excludes it.

In my understanding of Domo there is no way to disable the "Legend Item Toggle". However, I will start a ticket to see if I'm missing something.

[joanneesteban]

Thanks for sending that ticket over! I'm a little confused why the automatic thing is to exclude the data.

Also, weekly metrics over a 13 month span might be too convoluted - I automatically filter it down to a few months when I look at the dashboard, so I wonder if that's what the users would do when we demo.

[joanneesteban]

Notes from Education Demo

3/9/20

Questions Stakeholders Asked

• What does CSAT mean? Where does it come from?
• What would you like us to do with [the data that we are showing them]?
• Can I get access (when access was already provisioned)?
• Can we export & share the dashboards?
• Can we dig deeper into the date ranges?

Dashboard Features

Depending on if they will use the product, consider these iterations:

• Annotations: we're missing some key annotations that can help the users utilize the dashboards without questioning where the data is coming from (data accuracy & trust)
  • The URLs for the CSAT scores are still missing & we should add which type of surveys (5-star) in the Appendix or somewhere more obvious. I don't think the title hover-over is that intuitive.
  • Adding GA annotations - can we automate this? (i.e. spikes due to emails)
  • Adding follow up annotations: (i.e. what do we if something is below the target line.) Making it clear what our suggestions are.
    • Within this (& we can talk more about this as well), having both the technical and product side. Technical: look to see if a backend system was down, look to see if there's a bug on your frontend, etc. Product: is this a bug that your team will need to reprioritize your sprint to attend to? Is this something you should look into reprioritizing your roadmap to address or changing or attending to your KRs?
• Some users aren't receiving emails when they are getting added.

Product & Design

• Creating a short how-to for some basic usage? Potentially adding exporting, sharing, & date range drill downs
• Adding the context for using the dashboards to documentation outside of the Tool's product outline - somewhere that's user-facing

[amycesal]

Filtering in the chart is called the "Legend Item Toggle". I agree that it's super confusing. It's hard to use, and the experience is the opposite of what you expect. It's not possible to turn it off. I have a ticket in with our CSM about creating the ability to disable this feature because it's so confusing. I haven't heard back from him about it, so maybe it's something that we push on when we talk with him.

[jonwehausen]

Foresee DOMO Connector Block

• For the Facility Locator MVP, the survey we are attempting to retrieve data for is the "VA.gov Stars Only survey" -- which is of type "Feedback Survey"
• In attempting to automatically pull in data from this survey into DOMO, survey's of type "Feedback Survey" do not seem available
• There's two types of measurement data within the DOMO connector -- "Measure Data" and "Measure Definition"
• Neither of these options seem to populate with the ability to connect to the "VA.gov Stars Only" survey
• We either need to work with Foresee support in making this data available thru the "Measure Definition" or "Measure Data" API endpoints, or request DOMO support to add the ability to connect to the endpoint that surfaces the feedback survey types.

@joanneesteban

[amycesal]

Synthesis from the feedback survey

https://app.mural.co/t/adhocvetsgov9623/m/adhocvetsgov9623/1584995105129/553db2cbf58515f96db7c9639299a96de07ccb7c

High level takeaways

• all users said it was helpful or moderately helpful
• most people still wanted to also have the google sheet MVP available (only one 2, 2 maybe's and 1 yes)
• Monthly or weekly were both preferred, so maybe report weekly, with monthly labels
• contact center data was reported as not useful. However only one data point for contact centers was available, so more data might change that.
• Users don't know what to do with the data or how to make changes based on it. This was brought up in 2 different responses.
• the column headers could be better worded and less technical