Discuss how to break up the computedlabel tests in wpt/accname (to verify AccName computation)

cookiecrook commented 1 year ago

Discuss how to break up the automated computedlabel tests in wpt/accname to verify AccName computation related to testdriver implementation.

See also the resource utilities in wpt/wai-aria/scripts/aria-utils.js after WPT PR #38925 lands.

cookiecrook commented 1 year ago

@jaragunde I see you added a lot of label tests to Chromium recently (video). I wanted to make sure you're aware that many of these can be run in WPT now. Please join the discussion with ARIA WG AccName meetings and/or watch the https://github.com/web-platform-tests/interop-2023-accessibility-testing/issues repo.

spectranaut commented 1 year ago

some links:

chromium accname tests: https://source.chromium.org/chromium/chromium/src/+/main:content/test/data/accessibility/accname/

wpt accname tests: https://github.com/web-platform-tests/wpt/tree/master/accname/manual

cookiecrook commented 1 year ago

@spectranaut I don't believe we should worry about the WPT manual accname tests for this effort. Let's focus on the new automation capability.

spectranaut commented 1 year ago

heyy sorry I'm a bit confused, I really don't understand what you are asking.

You want to write tests for accname using the webdriver computedlabel in WPT, correct? And this issue is to discuss a plan for how to do it?

If so, we could start by transforming existing tests in chrome or the existing "manual" tests in WPT, but maybe you mean we should write them from scratch?

cookiecrook commented 1 year ago

@spectranaut wrote:

transforming existing tests in chrome

This makes sense to me and is part of why I copied in @jaragunde.

or the existing "manual" tests in WPT

I think we should review those as a last step to check for edge cases we don't catch in the first round of automated testing.

But in general, I think we could aim for:

Some balance between a single gargantuan test file and the ~160 test files in the manual directory.
One or more files that test the "happy path" cases for the overall algorithm.
An edge cases directory (or file name prefix) for testing any of the label tests not included in the main accname computation "happy path" tests (some of the manual tests could be ported here)
Perhaps separate directories (or file name prefixes) for the more difficult to test scenarios, like web components or other shadow DOM examples, the CSS cascade for generated alt fallback, etc.

Then again, if there aren't any strong opinions, I'll just start adding tests in the format how they make the most sense to me. If so, we can close this issue.

cookiecrook commented 1 year ago

Here's where I'm headed, based on the permalink IDs in the algorithm. Each heading below will be an individual test file. The No-Ops will not be tested.

comp_init (No-Op, No-Test) comp_computation (No-Op, No-Test)

comp_hidden_not_referenced.html

comp_labelledby.html

comp_labelledby_reset
comp_labelledby_foreach
comp_labelledby_set_current
comp_labelledby_recursion
comp_labelledby_append
comp_labelledby_return

comp_embedded_control.html

comp_embedded_control_textbox
comp_embedded_control_combobox_or_listbox
comp_embedded_control_range
comp_embedded_control_range_valuetext
comp_embedded_control_range_valuenow
comp_embedded_control_range_host_language_value

comp_label.html

comp_host_language_label.html

comp_name_from_content.html

comp_name_from_content_reset
comp_name_from_content_pseudo_element
comp_name_from_content_pseudo_element_before
comp_name_from_content_pseudo_element_after
comp_name_from_content_for_each_child
comp_name_from_content_for_each_child_set_current
comp_name_from_content_for_each_child_recursion
comp_for_each_child_append
comp_name_from_content_return

comp_text_node.html

comp_recursive_name_from_content.html

comp_tooltip.html

comp_append (No-Op) comp_complete (No-Op)

jaragunde commented 1 year ago

In case it's useful later, I created a list of the accname tests in Chromium and when they were added, so we can differentiate the ones that Joanmarie imported from the work group and the ones that were added later by Chromium devs. It also lists when the expectations were last modified, to know if the test needed changing from the original and why.

Info here: https://docs.google.com/spreadsheets/d/1cXaNgsraAQwVAfYPMuaeayoyAG-5pnztrMjewrK0mpY/edit?usp=sharing

w3c / accname