Some issues are difficult to unit tests, for example, on #1419, we went under 90% test coverage fixing an issue with keyboard nav, but it is not reasonable to test it in jest, so it was ignored. If we want to catch bugs like it more easily, we need a mechanism to run integration tests on a headless browser. Pa11y does this, but it is not configurable enough through its json file to meet all of our needs. We propose that we start running pa11y through its js interface, in conjunction with a test runner.
Acceptance Criteria:Create a list of functional outcomes that must be achieved to complete this issue
[ ] Pro/Con lists for available tools
[ ] Proof of concept of test runner tool which leverages our existing Pa11y tests
[ ] It is clear how we can continue to extend our test runner tool for future tests
[ ] All existing pa11y tests pass after conversion
[ ] Testing Checklist has been run and all tests pass
[ ] README is updated, if necessary
Tasks:Create a list of granular, specific work items that must be completed to deliver the desired outcomes of this issue
[ ] Create a script that runs pa11y through js api
[ ] integrate with test runner like mocha, chai, or ava
[ ] expose browser
[ ] provide a single test to demonstrate keyboard pa11y/integration testability (1419 may be a good place to start on this)
[ ] Run Testing Checklist and confirm all tests pass
Notes:Add additional useful information, such as related issues and functionality that isn't covered by this specific issue, and other considerations that will be helpful for anyone reading this
Description:
Some issues are difficult to unit tests, for example, on #1419, we went under 90% test coverage fixing an issue with keyboard nav, but it is not reasonable to test it in jest, so it was ignored. If we want to catch bugs like it more easily, we need a mechanism to run integration tests on a headless browser. Pa11y does this, but it is not configurable enough through its json file to meet all of our needs. We propose that we start running pa11y through its js interface, in conjunction with a test runner.
Acceptance Criteria: Create a list of functional outcomes that must be achieved to complete this issue
Tasks: Create a list of granular, specific work items that must be completed to deliver the desired outcomes of this issue
Notes: Add additional useful information, such as related issues and functionality that isn't covered by this specific issue, and other considerations that will be helpful for anyone reading this