w3c / aria-at

Assistive Technology ARIA Experience Assessment
https://aria-at.netlify.app
Other
154 stars 28 forks source link

Conflicting results for test 11 "Navigate to an expanded disclosure button from a link in the associated dropdown" (Disclosure Navigation Menu Example, V24.07.31) #1100

Closed tactics2 closed 33 minutes ago

tactics2 commented 3 months ago

Description of Behavior

Test Setup

Review Conflicts for "Navigate to an expanded disclosure button from a link in the associated dropdown"

  1. Assertion Results for "Control+Option+Left Arrow, then Control+Option+Left Arrow" Command and "List boundary is conveyed" Assertion

    • Tester tactics2 recorded output "- list, 4 items, level 2 - about, expanded button" and marked assertion as failing.
    • Tester IsaDC recorded output " list 4 items level 2 About expanded button You are currently on a button. To click this button, press Control-Option-Space." and marked assertion as passing.
  2. Unexpected Behaviors for "Shift+Tab" Command

    • Tester tactics2 recorded output "about, expanded button, list, 3 items" and noted "Other (Details: incorrectly announces that the list has 3 items instead of 4, Impact: MODERATE)".
    • Tester IsaDC recorded output " About expanded button list 3 items" and noted no unexpected behavior.

I marked it as failing because there is no definition of "list boundary is conveyed" in the test parameters. It is announced as a list, but what is a "boundary"?.

It also fails because it identifies the incorrect number of list items, but that is not in the test criteria, only that it announces it as a list. So technically it passes, even though it outputs incorrect information. IsaDC recorded the same results, but didn't mark it as an unexpected behavior in the comments.

mcking65 commented 33 minutes ago

resolved