Desk research for best practices/trends/models in design system research

[jaclinec]

In an effort to define a research process for conducting regular usability testing for USWDS components and patterns, a good first step is to do some desk research to get more familiar with best practices and possible approaches.

Purpose of desk research:

• Explore best practices for usability testing of components and patterns in a design system. What have others done? What models and approaches are recommended?
• Approaches for in-house component/pattern testing
• Approaches for putting a callout for users of design system to do their own testing and share results with us
• Considerations for testing with disabled participants

[jaclinec]

I'm documenting the desk research here: https://docs.google.com/document/d/1qkkqgGksoyJMqgAj73V_bfgpQPxb6_tBO2YuAjKi1Y8/edit?usp=sharing

[jaclinec]

Finished final edits and the report is complete:

Findings summary Ways to approach collecting data on components and patterns: Test complex interactions/patterns and components in-house by creating fictional scenarios and tasks for the tests where the components are meant to be interacted with. Solicit your community to share their research findings. Make it easy for the community to share: Consider adding an “ask” at every monthly call asking for them to share their research. Add requests in #uswds-public Slack channel and Github discussions from time to time. Consider asking for agency teams’ research findings on our website on relevant components and patterns pages. See example from Gov.uk design system:

In-house usability testing - general notes: Research should happen early and throughout the design process. Possible methods for early research include landscape analysis, interviews/workshops with stakeholders, interviews with users, surveys, and more. Usability testing should happen regularly and iteratively. For USWDS, plan for quarterly testing and test multiple patterns/components together. Determine how to prioritize what to test. Work out criteria you will consider when prioritizing. Consider several factors such as:

• reducing harm
• popularity of components/potential reach
• components affected by changes in WCAG guidelines
• What your community says - Invite your community to weigh in through Github discussions or surveys.

Communicate and be transparent about testing/research done. Publicly share testing results on relevant components and patterns pages and monthly calls. Make it clear which components need more research (gov.uk has an ‘experimental’ tag for instance).

Accessibility-first usability testing: Best practices

Ensure a baseline level of accessibility before usability testing (appropriate color contrast, alt text, etc.). Don’t waste time discovering problems in usability testing that could’ve been found with other, lower effort testing, such as:

• Standards reviews
• Heuristic evaluations
• Design walkthroughs
• Screening techniques

Like UX research, accessibility should be embedded at every stage of the design process. This “shifting left” approach means that projects should be budgeted with accessibility expertise included in all phases of the project as an essential ingredient; not an expendable component.

An intentional recruiting strategy for users with disabilities is essential. Aim to recruit with diversity in mind if it suits your study. Don’t just recruit screen reader users, consider also recruiting people with cognitive disabilities and situational disabilities (for example, those in rural areas with poor wifi).

Finding users with disabilities: Consider using an external recruiting firm, especially those that have good representation of people with disabilities on their research panel Make key contacts by looking in the right places:

• Social media groups
• Organizations for specific disabilities or conditions
• Cross-disability organizations
• Mailing lists
• College and university programs for students with disabilities
• Local disability-related support groups
• Local or regional government rehabilitation or disability services departments
• Seniors organizations and local senior centers
• Independent living organizations

Plan for additional time to recruit.

Tech prep and logistics planning is a MUST. Use pilot tests to work out kinks, such as issues that crop up with technology, considerations for logistics, facilitation, etc. Do tech checks prior to testing. Build in extra time at the beginning for breaking the ice, tech setup and config, and allowing them time to get familiar with the product before testing. Ensure you are able to record both human speech and the audio/speech of the AT (such as screen readers).

Do’s

• Plan for additional time. Consider the disability when working out timing.
• Be aware of energy levels and allow them to take as many breaks as needed. Prepare to end the session early if needed.
• It’s best to let them use their own equipment. Users of assistive tech often have the configurations the way they want and know their tech.
• Try to avoid asking users to change their configurations. For example, seasoned users of screen readers may have the speed rate turned high, making it difficult for the researcher to understand it. But, asking them to slow it down will affect data on task completion times, and also could frustrate the user. Only ask them to change it if absolutely necessary.
• Use high fidelity prototypes. Since screen readers rely on code to tell the user what’s on the page, low to medium fidelity prototypes are generally much harder to test with screen readers users.
• Use accessible technology to conduct the test, such as Zoom, and ensure it is compatible with their assistive tech.

Special considerations for users with cognitive disabilities:

• Provide instructions both written and verbal.
• If they have a caretaker, ensure the caretaker understands the goals and structure of the test and what to expect.
• Be prepared to break tasks into smaller sub-tasks to support memory/attention challenges or fatigue.
• Prepare to go off-script or modify tasks on the fly if needed.

Next steps

• Ideate and create a draft proposal of where research fits into each stage of the component and pattern lifecycle.
• Create a usability testing process proposal, incorporating accessibility best practices.
• Consult with stakeholders to begin defining a process for soliciting agencies to share relevant research findings with USWDS.
• Begin conversations with stakeholders on how USWDS can publicly share in-house testing results so that we are being transparent and building user trust.

General References Inclusive Patterns Team’s USWDS Lifecycle Management & Governance document How Gov.uk Design system prioritizes what components and patterns to work on How to research components and design patterns UX Mastery Discussion forum How do You Test a Design System A GUIDE TO DESIGN SYSTEM FIELD STUDIES Usability testing of design system components and patterns

Accessibility References What is Shift Left? (internal Bixal resource) Bixal Library Web Component User stories (internal Bixal resource) Real-Time Remote Usability Testing with Screen Reader Users, Part 1: Practical Overview Conducting my first ever remote accessibility user testing Accessibility in User-Centered Design: Evaluating for Accessibility Tips for recruiting people with disabilities Tips For Conducting Usability Studies With Participants With Disabilities How to Conduct Usability Studies for Accessibility DIFFERENCE BETWEEN USABILITY AND ACCESSIBILITY TESTING