VA Notify - Internal Ticket for Tracking

[michelle-dooley]

Issue Description

Internal team ticket for tacking VA Notify dashboard update request.

Data Lead: Julie Domo Lead: Pavan Support: Chloe & Michelle

---

Tasks

• [ ] _What work is necessary for t

Acceptance Criteria

• [ ] What will be created or happen as a result of this story?

  How to configure this issue

• [ ] Attached to a Milestone (when will this be completed?)
• [ ] Attached to an Epic (what body of work is this a part of?)
• [ ] Labeled with Team (product support, analytics-insights, operations, service-design, Console-Services, tools-fe)
• [ ] Labeled with Practice Area (backend, frontend, devops, design, research, product, ia, qa, analytics, contact center, research, accessibility, content)
• [ ] Labeled with Type (bug, request, discovery, documentation, etc.)

[pavanhothi]

Added new landing page URLs to BQ view per the KPI Dashboard Product Details doc for ticket #42378. Data is updated in Domo now as well. Created new card for the new campaign landing page. Will add to dashboard after @jonathan-epstein13 is able to do a QA on the data and card.

[pavanhothi]

@jonathan-epstein13 FYI new landing page campaign name is "10-10 EZ Resume Application"

[jonathan-epstein13]

Here is the GA segment to correlate to the card @pavanhothi created. Looks to line up pretty well. @jennaouellet maybe you can take a quick look to see if this segment makes sense. (I know I see a total of 51 users in GA vs 52 (unique) users in Domo in the period of 1/3/22-1/7/22.)

Couple things I noticed:

• The campaign(s) attached to the URL (https://va.gov/health-care/apply/application/resume) are not labeled '10-10 EZ Resume Application" in GA, as you can see here, so maybe we need to reach out in regards to that? (I know the campaign is in BQ, so we have that covered.)
• Also I noticed the URL https://va.gov/my-va/ was added to the VANotify google doc wondering if this was a landing page also attached to this specific campaign or it was for something else?

Adding @chloedotbrown note from slack thread: If they anticipate adding lots of different landing pages moving forward, I wonder if we should be rethinking how we're visualizing them. Perhaps we should experiment with a single, larger graph and add another card-level filter for landing pages (similar to the campaign one?). They'll still need us to update the data source, but at least that way, we won't have a growing number of landing page charts on an already crowded section. cc: @michelle-dooley @pavanhothi

[pavanhothi]

Thanks for the feedback @jonathan-epstein13!

• So the campaign name ("10-10 EZ Resume Application") attached to the new URL in Domo was purposely named that way by me by making an edit in the BQ view. This was done because prior to that change, we were getting two different campaign names populating for the same new URL, the ones which you pointed out are showing up in GA. So I edited the campaign name for consistency, and chose this specific name since that was what they had the form name listed as in the Google doc. I'll comment on their ticket to ask and confirm if this campaign name works for them.
• I also noticed the second URL they had listed on the Google doc, which I found to be completely unrelated to the new Form/campaign. My understanding was that it is a new URL to be added, but it is not attached to this new campaign. I added it in the BQ view also, just did not attach it to the new campaign name if that makes sense. I can comment on their ticket to inquire and confirm this as well.
• I can experiment with a new single larger graph in Domo so that we don't need to keep adding all these new cards to that section. I will update again once I have something created for you to take a look at and QA.

[pavanhothi]

@jonathan-epstein13 Just got a reply back on the main ticket. Looks like they are good with the campaign name we have created. And looks like the second URL was related to a different campaign as we had thought. Will update now once I create a new single larger graph card. https://github.com/department-of-veterans-affairs/va.gov-team/issues/42378#issuecomment-1150464628

[pavanhothi]

@jonathan-epstein13 just shared out the new single card I created with all five (four current and one new) of the the VANotify landing pages on it. Please let me know what you think. Thank you!

[jonathan-epstein13]

@pavanhothi the new VANotify landing page card looks great! I know we will have to update it (when/if the VANotify team wants more landing pages added, but it has all landing pages in one graph, similar to how it was before.) I like how you can select/deselect the landing page you can see within the graph. It is a great option to see the data clearly.

[michelle-dooley]

@pavanhothi - I like the all in one graph approach I guess my only struggle is when there is no landing page filter selected it is hard to read with such different scales between them. With that said I don't know how often or at all they will view all together or if they will only be looking by campaign filter which is a much easier to read view.

@chloedotbrown - what do you think?

@jonathan-epstein13 - are we good with QA or do we still need @jennaouellet to weigh in?

[jonathan-epstein13]

@michelle-dooley working on making sure the landing page section matches up to what is in GA. I might need @jennaouellet to just confirm I have the correct segments in place. The design QA is finished and the new card with the filter campaign runs smoothly.

[jonathan-epstein13]

GA-QA (Landing Pages) Findability
Chart title	Assets	Pass / Fail
Claims & Appeals - VANotify users	Custom segment	🟢 PASS
Manage VA Debt - VANotify users	Custom segment	🟢 PASS
VA Profile - VANotify users	Custom segment	🟢 PASS
Add or Remove Dependents - VANotify users	Custom segment	🟢 PASS
Apply for VA Healthcare	Custom segment	🟢 PASS
Avg. session per user for VANotify source	BigQuery metric	🟢 PASS
Bounce rate for VANotify sessions	Custom segment	🟢 PASS

[michelle-dooley]

@jonathan-epstein13 @chloedotbrown @pavanhothi - so is this one complete then, are we ready to let the VANotify team their updates have made?

[jonathan-epstein13]

@michelle-dooley on the Data QA side, this is complete as posted previously. I thought I attached the Design QA, so I'll post it here.

Design QA for Landing pages. Test	Expected Behavior	Actual Behavior	Pass/Fail
Dashboard title	Text fully readable, without errors	Text fully readable, without errors	Pass ✅
Date filter	Every chart gets filtered for date and group by settings are changed	Every chart gets filtered for date and group by settings are changed	Pass ✅
Dashboard filter	Every card gets filtered by the dashboard level filter(s)	Every card gets filtered by the dashboard level filter(s)	Pass ✅
Section titles	Every section title & subtitle are spelled correctly	Every section title & subtitle are spelled correctly	Pass ✅
Chart titles	Every chart title is fully readable, without errors, relevant to chart	Every chart title is fully readable, without errors, relevant to chart	Pass ✅
Chart descriptions	All chart descriptions are disabled, so nothing appears when hovering over title (except title, but no different text)	All chart descriptions are disabled, so nothing appears when hovering over title (except title, but no different text)	Pass ✅
Hover text	All hover text "makes sense" and is spelled correctly	All hover text "makes sense" and is spelled correctly	Pass ✅
Annotations	Every annotation "makes sense," is free of errors, and visible within text box	Every annotation "makes sense," is free of errors, and visible within text box	Pass ✅
Text box formatting	Every annotation and section title box shows all text, does not generate scroll bar	Every annotation and section title box shows all text, does not generate scroll bar	Pass ✅
Card interactions – filters	Clicking data point does not filter other cards	Clicking data point does not filter other cards	Pass ✅
Card interactions – card detail	Clicking chart title opens card in new tab	Clicking chart title opens card in new tab	Pass ✅
Display settings – multi-value gauges	Date grouping ("by month") appears under each title	Date grouping ("by month") appears under each title	Pass ✅
Display settings – summary numbers	All tables, bar charts, and filled gauges show date range	All tables, bar charts, and filled gauges show date range	Pass ✅
Related cards	In card detail view, all related card previews have been deleted	In card detail view, all related card previews have been deleted	Pass ✅
Appendix	No charts or notecards appear in appendix below dashboard	No charts or notecards appear in appendix below dashboard	Pass ✅

[michelle-dooley]

This work is complete.