jilladams
commented
2 months ago This came up on https://github.com/department-of-veterans-affairs/vets-website/pull/31801, where eventually the flaky test didn't flake / PR could merge but: it was annoying, so we decided to ticket since this could happen to any vets-website PR where the test fails.
User Story or Problem Statement
In the Facility Locator, we have a flaky Axe test that blocks PRs.
Description or Additional Context
WCAG has a rule that any interactive element within a scrollable div must have its own tab index, in order to allow users to tab to it. In the Facility Locator, our search results have interactive links that must be tabbable.
An automated test in vets-website uses Axe to check tab indexes, and sometimes fails:
src/applications/facility-locator/tests/e2e/providerSearch.cypress.spec.js. This test has turned up in the flaky test report (9/3), and when it fails on a PR, it blocks merge of that PR.We aren't sure if the axe console errors are the same as what the axe automated tests would be reporting. However, the axe console errors can't be replicated with the axe chrome extension. It's possible some of this may have to do with timing? For example, a console error that I (Laura) is seeing a lot is regarding the H1, but these pages do have an H1 but they also need to load that H1 because the page is an application rather than static content:
We aren't sure what to do about that, though. Removing axe is not the preferred option, because it does also catch legitimate errors.
Acceptance Criteria