QA Process improvements for tracking

jilladams commented 2 weeks ago

QA thoughtstarter docs:

Definitions

Usability Research during prototype phase. Rarely happens for builds headed to production.
Code review: between engineers
Automated testing: assessed and added by engineers
Design review
A11y review
Manual testing: Try to break it (A QA walks into a bar and orders -1 beers)
Production verification / User Acceptance Testing

Design review: AC to ensure it happens (invisible effort, not counted in points / velocity)
A11y review: AC to ensure it happens (invisible effort, not counted in points / velocity)
Product UAT as ACs
Automated testing:
- Refinement: explicit scoping / make it visible in ACs
Manual testing: Assessing the scale necessary for the implementation, and either:
- Refinement: explicit scoping / make it visible in ACs
- Planning:
- Adding to implementation effort with its own points or
- Creating separate QA tickets

Large projects:
- Deserve their own QA tickets, so we take the time we need. (OR: more cohesive sprint goals / collaboration?)
- Test plans are necessary.
- Shared effort to do the testing.
- Define / think about “happy path” when it pertains to multiple options, e.g. in Drupal
- Drupal > FE projects:
- Need to exercise the full range of Drupal data. Can be programmatically assisted. Also: manual data entry helps demonstrate the Editor user experience.
- FE: making sure all the data comes through, in the right place, in the right order, with the right conditionals, and a11y levels. This is QA, and needs its own time / estimates.
Small projects:
- Not just happy path eng > eng QA steps. Upside down / inside out: how could it break?

Purpose: Ensure code quality, adherence to standards, and identify potential bugs early.
Who: Developers, often in pairs or small groups.
When: Before merging code into the main branch (ideally after each small, logical change).
How: Manual inspection, sometimes aided by tools (static analysis, linters).
Best Practices:
- Use a collaborative, blameless approach.
- Focus on functionality, maintainability, and readability, DRY code.
- Automate basic checks with tools, and ensure automated test coverage (unit, integration, end-to-end)
- Ensure thorough thoughtfulness about blast radius and cross-product implications

Purpose: Test the software's functionality, performance, and security against defined requirements.
Who: Assigned QA for the feature / product enhancement, since we do not have a QAE that may be Engineers, UX folks, A11y, Product, or Delivery. (Basically: anybody can test.)
When: After incremental work is built and engineer has verified first pass at happy path, sad path, and basic use cases.
How: and manual exploratory testing.
Best Practices:
- Perform manual testing for edge cases and user experience.

Purpose: Validate that the software meets user needs and expectations.
Who: Product Manager, VA Product Owner/Lead, potentially end user Editors or business stakeholders, depending on the product / project
When: When feature / build is available to preview in Preview Server, Tugboat, or Staging, prior to production launch.
How: Manual testing based on user scenarios, sometimes using lower environments.
Best Practices:
- Follow HCD best practices to gather real user feedback that informs design and implementation.
- Create clear acceptance criteria for features.
- Use real-world data and environments for testing whenever possible.
- Make UAT a gatekeeper for releases

jilladams commented 2 weeks ago

jilladams commented 2 weeks ago

jilladams commented 1 day ago

Added the Code Review, QA, UAT notes / expectations to the Quality section of the team charter: https://docs.google.com/document/d/1_qowYScuIW617jkPeS0KFj35Oq79SW-LRHeTkRlnEOg/edit