Closed joanneesteban closed 3 years ago
Some initial feedback (using our Participant Test User account):
Topic | Notes |
---|---|
Exports | Formatting for exports to PPT are a little off: |
form_name Filter | When I go onto the dashboard, the title automatically populates to Higher Level Review. But the data reflects all of the Forms that are in the dataset. When I filter for ALL form_names, the title also says Higher Level Review, and when I filter for NONE` of the form_name's, the title says Higher Level Review and filters for all of the data as well. Is there a way to edit the title for "All available Forms" or something of the equivalent? |
Footer Links | The hyperlinks at the footer aren't clickable |
Compiled changes to improve usability:
Updated handling of dates in filters, graphs without date in X-axis, and units of comparison in multi-value gauges to be consistent with the new logic used in Search dashboard build (see comment here). This should make it easier for the user to understand how data is being compared and filtered.
@chloedotbrown sorry about the delay here.
The following was tested using our "Participant" user account.
Another random thing: does the chart selection for "successful submissions" in the highlights section make sense? It looks like there were submissions during dates where there weren't because of the area fill.
Test | Expected Behavior | Actual Behavior | Pass/Fail |
---|---|---|---|
Default page | Page does not automatically filter, and description reflects lack of filter. | Page does not automatically filter, and description reflects lack of filter. | Pass ✅ |
Filter on Forms | Every chart gets filtered | Every chart gets filtered | Pass ✅ |
Filter on Device Category | Every chart gets filtered | (Form 1010EZ filtered on "mobile") The following charts do not appear to have the device category filter in the filter hovers: Highlights section: - Completed in 1 session Form timing & exits section: - Avg. sessions to complete - Sessions to complete form |
Fail ❌ |
Filter on Browser | Every chart gets filtered | (Form 1010EZ filtered on "Firefox") The following charts do not appear to have the browser filter in the filter hovers: Highlights section: - Successful submissions - Forms by status - Unique users - Completed in 1 session Form completion section: - Forms started - Forms submitted - Form completion rate - Completion by device Form timing & exits section: - Avg. sessions to complete - Sessions to complete form Finding the form section: - Top pages viewed before accessing the form Landing page section: - Unique visitors - Avg. visits per user - Avg. seconds on page - Avg. scroll depth - Left without visiting form - Top sources - Top campaigns |
Fail ❌ |
Date Filters | Every chart gets filtered on date | - Top sources chart in How did people find the landing page section does not filter for any form filter and non-default date filter- (Form 5655 filtered on "This Month") Successful submissions chart in Highlights section filters additional date ranges (screenshot should filter from June 1) |
Fail ❌ second & third bullet points may be a Domo limitation. |
How to use this dashboard - header | Links to Platform Website, "How to use this dashboard" page | Links to internal Confluence page | Fail ❌ Need to update to Platform Website link |
How to use this dashboard - footer | Links to Platform Website, "How to use this dashboard" page | Links to internal Confluence page | Fail ❌ Need to update to Platform Website link |
Chart Title | Corresponding filtered form appears as the page title | Corresponding filtered form appears as the page title | Pass ✅ |
See form landing page | Links for all available Forms go to the correct VA.gov production landing page | Form 22-1995 goes to https://www.va.gov/education/how-to-apply/ instead of https://www.va.gov/education/apply-for-education-benefits/application/1995/introduction | Fail ❌ Need to fix 1995 landing page (double check data source as well) |
Form description | Corresponding filtered form description appears with every filtered form | Corresponding filtered form description appears with every filtered form | Pass ✅ |
Data sources | Correct data sources appear | Medallia appears as well as Google Analytics & ForeSee | Fail ❌ Is Medallia data included here? This will affect the annotation in User surveys section as well. "Foresee" should be "ForeSee" |
Section titles | Every section title & question is spelled correctly | Every section title & question is spelled correctly | Pass ✅ |
Chart titles | Every chart title is spelled correctly | Every chart title is spelled correctly | Pass ✅ |
Annotations | Every annotation "makes sense" and is spelled correctly | Every annotation "makes sense" and is spelled correctly | Pass ✅ |
Labels | All data labels appear where expected. | All data labels appear where expected. | Pass ✅ |
Hover text | All hover text "makes sense" and is spelled correctly | Highlights section: - Forms by status title hover is "test description" |
Fail ❌ |
Funnel hover text | All funnel hover text "makes sense" and is spelled correctly for each form | (Form 20-0996 filtered) 7 - Review and Submitt should be 7 - Review and Submit |
Fail ❌ |
VA.gov Analytics Playbook link | Links to Github Analytics Playbook | Links to Github Analytics Playbook | Pass ✅ |
Google Analytics link | Links to Google Analytics 360 | Links to Google Analytics 360 - WBC | Pass ✅ |
VA.gov Data Dictionary link | Links to Platform Website, "Forms Data Dictionary" page | Links to internal Confluence page | Fail ❌ Need to update to Platform Website link |
Domo Help Center link | Links to Domo Help Center | Links to Domo Help Center | Pass ✅ |
@joanneesteban The introduction page for 22-1995 is step 1 of the funnel. Does it make sense to have the landing page be the same as step 1? Or would https://www.va.gov/education/change-gi-bill-benefits/ work better?
@JasonTheBearded was that intro page defined by the Education team? If so, then we can probably keep the landing page that's currently on the dashboard. Otherwise, the link that you have is probably better since the landing page can be the start to multiple education forms.
Filter on device category/browser: Failed filter tests caused by those fields not being included/named differently in some of our datasources. We've added both to Standard Forms User Completion and browser to Standard Forms Engagement and Standard Forms Landing Pages - will retest once updated datasets have reloaded. Date filters:
Data sources: Have removed "Medallia" from list of data sources in dashboard intro text and user survey section description. "S" in ForeSee now capitalized in both locations. Medallia task completion placeholder also removed. I've also added language to call out that device and browser filters don't work on the user survey data, which could be added to the data dictionary as well.
Hover text: "Test description" removed from card description field. No hover text appears above title for "Forms by status" card in the Highlights section.
Funnel hover text: Wondering what the best approach to fixing this typo is. @JasonTheBearded – could we correct in an ETL or is there another upstream solution?
I think most of the other failures are link updates once documentation is published on the Platform site, which we can dig into next week.
Fantastic! Thank you both for the quick fixes here! 🎉
@chloedotbrown For the funnel hover text it's an upstream fix, the step name was inserted into the table incorrectly. i've fixed that and am refreshing the datasource
@chloedotbrown Data refresh complete for funnel and spelling error is corrected. Good to go :)
Double-checked device and browser filters and funnel text, now that all data has been refreshed. All passes! ✅
Going through the QA checklist again. Is this chart supposed to be blank, @chloedotbrown @JasonTheBearded ?
It's also blank for form 5655.
@chloedotbrown @JasonTheBearded almost there!
Test done using the Participant Tester:
Test | Expected Behavior | Actual Behavior | Pass/Fail |
---|---|---|---|
Default page | Page does not automatically filter, and description reflects lack of filter | Page does not automatically filter, and description reflects lack of filter | Pass ✅ |
Filter on Forms | Every chart gets filtered for every form | "Top pages viewed before accessing the form" in Finding the form section is blank for 526 and 5655 |
Fail ❌ |
Filter on Device Category | Every chart gets filtered on device category | Every chart gets filtered on device category | Pass ✅ |
Filter on Browser | Every chart gets filtered on browser | Every chart gets filtered on browser | Pass ✅ |
Date Filters | Every chart gets filtered on date | Every chart gets filtered on browser | Pass ✅ |
How to use this dashboard - header | Links to Platform Website, "How to use this dashboard" page | Links to Platform Website, "How to use this dashboard" page | Pass ✅ |
How to use this dashboard - footer | Links to Platform Website, "How to use this dashboard" page | Links to Platform Website, "How to use this dashboard" page | Pass ✅ |
Chart Title | Corresponding filtered form appears as the page title | Corresponding filtered form appears as the page title | Pass ✅ |
See form landing page | Links for all available Forms go to the correct VA.gov production landing page | Links for all available Forms go to the correct VA.gov production landing page | Pass ✅ |
Form description | Corresponding filtered form description appears with every filtered form | Corresponding filtered form description appears with every filtered form | Pass ✅ |
Data sources | Correct data sources appear | Correct data sources appear | Pass ✅ |
Section titles | Every section title & question is spelled correctly | Every section title & question is spelled correctly | Pass ✅ |
Chart titles | Every chart title is spelled correctly | Every chart title is spelled correctly | Pass ✅ |
Annotations | Every annotation "makes sense" and is spelled correctly | Every annotation "makes sense" and is spelled correctly | Pass ✅ |
Labels | All data labels appear where expected. | All data labels appear where expected. | Pass ✅ |
Hover text | All hover text "makes sense" and is spelled correctly | "Forms by status" in Highlights title annotation says, "Test description" |
Fail ❌ |
Funnel hover text | All funnel hover text "makes sense" and is spelled correctly for each form | All funnel hover text "makes sense" and is spelled correctly for each form | Pass ✅ |
VA.gov Analytics Playbook link | Links to Github Analytics Playbook | Links to Github Analytics Playbook | Pass ✅ |
Google Analytics link | Links to Google Analytics 360 | Links to Google Analytics 360 | Pass ✅ |
VA.gov Data Dictionary link - header | Links to Platform Website, "Forms Data Dictionary" page | Links to Platform Website, "Forms Data Dictionary" page | Pass ✅ |
VA.gov Data Dictionary link - footer | Links to Platform Website, "Forms Data Dictionary" page | Links to Platform Website, "Forms Data Dictionary" page | Pass ✅ |
Domo Help Center link | Links to Domo Help Center | Links to Domo Help Center | Pass ✅ |
@joanneesteban , thanks for catching! Looks like there was a rogue exclusion filter on the "Top sources" card. I've removed it, and both are now filtering correctly (screenshot below). I also removed the test description from "Forms by status"... and double checked all other cards for this. All seems clear, but feel free to give it another look!
Thanks for the rapid work, @chloedotbrown !
Update to the test above 06-29-2021 17:36 EST
Test done using the Participant Tester:
Test | Expected Behavior | Actual Behavior | Pass/Fail |
---|---|---|---|
Default page | Page does not automatically filter, and description reflects lack of filter | Page does not automatically filter, and description reflects lack of filter | Pass ✅ |
Filter on Forms | Every chart gets filtered for every form | Every chart gets filtered for every form | Pass ✅ |
Filter on Device Category | Every chart gets filtered on device category | Every chart gets filtered on device category | Pass ✅ |
Filter on Browser | Every chart gets filtered on browser | Every chart gets filtered on browser | Pass ✅ |
Date Filters | Every chart gets filtered on date | Every chart gets filtered on browser | Pass ✅ |
How to use this dashboard - header | Links to Platform Website, "How to use this dashboard" page | Links to Platform Website, "How to use this dashboard" page | Pass ✅ |
How to use this dashboard - footer | Links to Platform Website, "How to use this dashboard" page | Links to Platform Website, "How to use this dashboard" page | Pass ✅ |
Chart Title | Corresponding filtered form appears as the page title | Corresponding filtered form appears as the page title | Pass ✅ |
See form landing page | Links for all available Forms go to the correct VA.gov production landing page | Links for all available Forms go to the correct VA.gov production landing page | Pass ✅ |
Form description | Corresponding filtered form description appears with every filtered form | Corresponding filtered form description appears with every filtered form | Pass ✅ |
Data sources | Correct data sources appear | Correct data sources appear | Pass ✅ |
Section titles | Every section title & question is spelled correctly | Every section title & question is spelled correctly | Pass ✅ |
Chart titles | Every chart title is spelled correctly | Every chart title is spelled correctly | Pass ✅ |
Annotations | Every annotation "makes sense" and is spelled correctly | Every annotation "makes sense" and is spelled correctly | Pass ✅ |
Labels | All data labels appear where expected. | All data labels appear where expected. | Pass ✅ |
Hover text | All hover text "makes sense" and is spelled correctly | All hover text "makes sense" and is spelled correctly | Pass ✅ |
Funnel hover text | All funnel hover text "makes sense" and is spelled correctly for each form | All funnel hover text "makes sense" and is spelled correctly for each form | Pass ✅ |
VA.gov Analytics Playbook link | Links to Github Analytics Playbook | Links to Github Analytics Playbook | Pass ✅ |
Google Analytics link | Links to Google Analytics 360 | Links to Google Analytics 360 | Pass ✅ |
VA.gov Data Dictionary link - header | Links to Platform Website, "Forms Data Dictionary" page | Links to Platform Website, "Forms Data Dictionary" page | Pass ✅ |
VA.gov Data Dictionary link - footer | Links to Platform Website, "Forms Data Dictionary" page | Links to Platform Website, "Forms Data Dictionary" page | Pass ✅ |
Domo Help Center link | Links to Domo Help Center | Links to Domo Help Center | Pass ✅ |
All tests passed. Closing this ticket!
Issue Description
As we start to roll out our dashboards, we need to make sure that our dashboards are usable.
Tasks
QA for the following
Acceptance Criteria