Accessibility test standardization, improvements and documentation

[rahimabdi]

Tracker:

• [x] Share draft documentation with @boazsender @cookiecrook (done Apr 28 2024)
• [ ] Determine publication platform (likely our GitHub Wiki page)
• [ ] Determine/publish updates to related ARIA testing documentation
• [ ] Determine if we need to merge this existing ARIA WPT documentation into the new document: https://github.com/w3c/aria/blob/main/documentation/wpt.md

  ---

@cookiecrook Consolidating general thoughts here on improving the quality/consistency of our WPT accessibility test files and providing supporting documentation for test authors (also to make it less daunting for WPT contributors). All feedback welcome! Happy to take this forward after you and other folks have provided input.

I know that detailed guidance is already captured here and this may be moved to web-platform-tests/interop-accessibility: Running the WPT Tests for ARIA and Related Specs.

• Writing tests, guidance and implementation practices
  • Boilerplate comment header for all test files (e.g., "This file tests X...")
  • Providing documentation links to relevant parts of a spec being tested
  • Comment links to where relevant testing is located (and documentation scheme for referencing files in other locations)
  • Naming .html test files (usually after a spec's section)
  • Readability improvement guidance: e.g., alphabetization where possible, prescriptive order of element attributes (e.g., <div id="id" data-expectedtestname="someTest" data-expectedlabel="label">, spacing and indentation, etc.
  • Subtest names must be globally unique (the more descriptive the better)
  • id naming scheme for elements/containers (e.g., many elements with unique ids referenced via aria-labelledby)
  • Performing HTML validation
  • Using PR checks (primarily ensuring that lint is clean, removing whitespace)
  • PR commit message (description and co-author attribution)
• Supporting documentation for test authors/maintainers
  • Overview of the WPT project
  • Understanding/viewing CI results
  • More on testharness.js
  • Index page with all a11y related tests (e.g., accname, wai-aria, core-arm, html-aam, svg-aam, etc.)
  • WPT Debugging Q&A and common issues (e.g., troubleshooting bad data* attribute on test, non-unique test name, improper use of aria-utils utility functions)
  • Explainer for aria-utils.js and what it does

[cookiecrook]

Generally all sounds good. I think some of the non-accessibility-specific stuff you mentioned above may already be documented in the main WPT project... Any accessibility-specific docs could go here, along with pointers to the main WPT docs where relevant.

[boazsender]

@rahimabdi @lolaodelola and I met and discussed a few additional ideas:

• We talked about creating a guide for where to put tests
• Create an index page of all the tests using the tags
• Going back and tagging existing tests
• Linking to these docs from web-platform-tests.org

We also talked about our multiple audiences

• People talking about different accessibility tests (e.g. aria test writers)
• people in other groups who want to test accessibility

[cookiecrook]

Adding Agenda+ for docs discussion in the May meeting.

[rahimabdi]

@boazsender @cookiecrook I've emailed you (and Lola) a link to the draft documentation I'm proposing. Your thoughts and feedback are welcome!

I've also added a tracker to the issue description with outstanding tasks/questions.

[rahimabdi]

Documentation draft: https://docs.google.com/document/d/1Q3A8Y-xcqcKL2BUYyakroWDdHSiYQjkxtxzx_twR0GQ/edit?usp=sharing.

[cookiecrook]

@chrishtr you took an action to review. Thank you!

[foolip]

Determine publication platform (likely our GitHub Wiki page)

@chrishtr asked me, and I think most of that document would make sense in https://github.com/web-platform-tests/wpt/tree/master/docs/writing-tests, as a new accessibility.md file most likely. It would then be published to https://web-platform-tests.org/.

Parts of the document are about the Accessibility Interop Group, and that I'd suggest putting right here in this repo's README where it's easy to find or stumble upon.

Some of the FAQ questions could probably be put into the main WPT docs and some others are about the Accessibility Interop Group. Splitting that and the overall doc would be shame in a way since it's useful as a rather self-contained resource. An option could to publish it as a whole in this repo as well, but just note which parts are also available on https://web-platform-tests.org/ and which is the source of truth.

Hope that helps.

[rahimabdi]

Thanks for chiming in @foolip.

Splitting that and the overall doc would be shame in a way since it's useful as a rather self-contained resource. An option could to publish it as a whole in this repo as well, but just note which parts are also available on https://web-platform-tests.org/ and which is the source of truth.

Agreed, I would also vote for keeping it self-contained in this repo but could be split up otherwise.