Consider including browser makers in our process

[mfairchild365]

From Hadi:

While I have no problem recording my observation, I strongly believe that both screen reader and browser makers should be part of this process to see first-hand how the information is rendered and trace any potential bugs in their system so they can fix the issues as we go. I understand that we want to record our findings but I believe our main goal is to help them to produce the expected output. If it is the main goal, I think we can achieve it a lot sooner and easier if they become part of this testing process.

Maybe I haven’t understood the mission of this group but for me it is a typical collaboration that we have with various software companies. I always have them work and engage them in the testing process. It would be great if we can use a few minutes of our next meeting to talk why they are not engaged in this process.

We have already documented how AT developers are included in our process, but we do not appear to have any documentation around how we include browser developers in the process.

• Is there an opportunity to better include browser developers?
• Should we document a process that we should follow if a test fails due to a browser issue rather than an AT issue?

[mcking65]

We can add some documentation regarding where to raise browser-related bugs. In terms of process, though, aria-at is just one piece of the stack testing process. The scope of aria-at is the last mile from the operating system API to the user.

1. ARIA Working Group tests browser support, i.e., makes sure that each ARIA attribute is correctly reflected in operating system APIs.
2. APG Task Force provides the reference implementations of ARIA based on the ARIA specification.
3. ARIA-AT tests whether assistive technologies accurately render the reference implementations.

If we find a browser bug, that would indicate a failure in the ARIA testing. So, we would not only want to raise a browser bug but also examine the ARIA tests to ensure that it appropriately covers the relevant ARIA attribute.

When we are running aria-at tests, we do not expect the testers to do root cause analysis of a failure. However, we will have data that enables us to identify upstream causes that are either in the test case (reference implementation) or the browser. If a given assertion fails exactly the same way in multiple assistive technologies but only in a specific browser, that could point to a browser bug.

In other words, the formal channel for doing this is the ARIA Working Group, to which many of us belong. Of course, we are also directly involving members of the ARIA working group who represent each of the browser teams.