Create Platform QA Dashboard

pjhill commented 2 years ago

Problem Statement

OCTO-DE, Platform and VFS Leadership don't currently have a way to easily understand the state of quality across all products hosted on the platform.

How might we develop a solution that allows stakeholders to see QA-related metrics from the product(s) that are relevant to them across the platform.

Hypothesis or Bet

By providing a dashboard that enables stakeholders to visualize quality assurance practice metrics for products all over the platform, stakeholders will have better insight into the state of quality on the platform and will be able to make better decisions to address any gaps in quality for products.

We will know we're done when... ("Definition of Done")

Users can view quality assurance metrics for all products on the platform.

Known Blockers/Dependencies

Requires a Complete Product List

Projected Launch Date

Q2 2022

Launch Checklist

Guidance (delete before posting)

This checklist is intended to be used to help answer, "is my VSP initiative ready for launch?". All of the items in this checklist should be completed, with artifacts linked---or have a brief explanation of why they've been skipped---before launching a given VSP initiative. All links or explanations can be provided in Required Artifacts sections. The items that can be skipped are marked as such.

Keep in mind the distinction between Product and Initiative --- each Product needs specific supporting documentation, but Initiatives to improve existing Products should reuse existing documentation for that Product. VSP Product Terminology for details.

Is this service / tool / feature...

... tested?

[ ] Usability test (TODO: link) has been performed, to validate that new changes enable users to do what was intended and that these changes don't worsen quality elsewhere. If usability test isn't relevant for this change, document the reason for skipping it.
- [ ] ... and issues discovered in usability testing have been addressed.
- Note on skipping: metrics that show the impact of before/after can be a substitute for usability testing.
[ ] End-to-end manual QA or UAT is complete, to validate there are no high-severity issues before launching
[ ] (if applicable) New functionality has thorough, automated tests running in CI/CD

... documented?

[ ] New documentation is written pursuant to our documentation style guide
[ ] Product is included in the List of VSP Products
- List the existing product that this initiative fits within, or add a new product to this list.
[ ] Internal-facing: there's a Product Outline
[ ] External-facing: a User Guide on Platform Website exists for this product/feature tool
[ ] (if applicable) Post to #vsp-service-design for external communication about this change (e.g. VSP Newsletter, customer-facing meetings)

... measurable

[ ] (if applicable) This change has clearly-defined success metrics, with instrumentation of those analytics where possible, or a reason documented for skipping it.
- For help, see: Analytics team
[ ] This change has an accompanying VSP Initiative Release Plan.

When you're ready to launch...

[ ] Conduct a [go/no-go] (https://vfs.atlassian.net/wiki/spaces/AP/pages/1670938648/Platform+Crew+Office+Hours#Go%2FNo-Go) when you're almost ready to launch.

Required Artifacts

Documentation

PRODUCT_NAME: directory name used for your product documentation
Product Outline: link to Product Outline
User Guide: link to User Guide

Testing

Usability test: link to GitHub issue, or provide reason for skipping
Manual QA: link to GitHub issue or documented results
Automated tests: link to tests, or "N/A"

Measurement

Success metrics: link to where success metrics are measured, or where they're defined (Product Outline is OK), or provide reason for skipping
Release plan: link to Release Plan ticket

TODOs

[ ] Convert this issue to an epic
[ ] Add your team's label to this epic

mchelen-gov commented 2 years ago

@JoeTice What is the status of this Initiative?

annekerr49 commented 2 years ago

@JoeTice - Joe we are trying to increase the information available to those persons who monitor this board. To that end we are asking that open items updated monthly as to any changes. For our interested colleagues. Please let me know if you have any questions.

JoeTice commented 2 years ago

Hi @annekerr49 (also @mchelen-gov ) - Sorry, I didn't get a notification on this...

The status of this effort is 'Not Started', It was initially blocked by the launch of the VFS Product Directory, which launched last quarter (Q3 2022 by calendar quarters, Q4 2022 by govt quarters). With that roadblock resolved, this effort could be resumed - although, we would also recommend considering doing this other effort prior...

https://github.com/department-of-veterans-affairs/va.gov-team/issues/44735

This effort (which is fully planned and has its tasks fully detailed in Zenhub ), would lay the groundwork for everything that would be needed in the 'Platform Dashboard'.

Essentially - doing a 'Product Dashboard' for every VFS product first. Then rolling those results into a 'Platform Dashboard' that would show the high level view of all VFS Products. We felt that this would be a much more effective flow than trying to do the Platform Dashboard first. A lot of the groundwork has already been created for the Product Dashboards, so that it would be a quick ramp up after the VFS Product Directory launched.

While the Platform Dashboard's ticket was never refined to reflect the current state or the future work - the Product Dashboard tickets are fully detailed and up to date, reflecting what would need to be done.

Would be happy to get on a call to discuss if that would be helpful at any point.

EricaRobbins commented 11 months ago

Needs re-evaluated, connect with Peter H. - @AparnaNittalaUSDS Obtain more clarification

annekerr49 commented 8 months ago

Aparna followed up with Joe. We know what we want to achieve. Joe has action item to work with Joe Hill to update epic.
Some discovery phase efforts.

department-of-veterans-affairs / va.gov-team