Possible incorrect results for `Values assigned to role="" are not valid ARIA roles.` ARIA check in Lighthouse audit

hamirmahal commented 1 month ago

Product

axe-core

Product Version

"^4.9.0"

Latest Version

[X] I have tested the issue with the latest version of the product

Issue Description

Expectation

Reported ARIA role violations should actually be violations.

Actual

The ARIA check seems to report failures on a valid ARIA role, at least according to https://dequeuniversity.com/rules/axe/4.8/aria-allowed-role.

How to Reproduce

body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-top
<ul role="region" aria-live="polite" id="chakra-toast-manager-top" style="position: fixed; z-index: var(--toast-z-index, 5500);">
body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-top-left
<ul role="region" aria-live="polite" id="chakra-toast-manager-top-left" style="position: fixed; z-index: var(--toast-z-index, 5500);">
body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-top-right
<ul role="region" aria-live="polite" id="chakra-toast-manager-top-right" style="position: fixed; z-index: var(--toast-z-index, 5500);">
body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-bottom-left
<ul role="region" aria-live="polite" id="chakra-toast-manager-bottom-left" style="position: fixed; z-index: var(--toast-z-index, 5500);">
body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-bottom
<ul role="region" aria-live="polite" id="chakra-toast-manager-bottom" style="position: fixed; z-index: var(--toast-z-index, 5500);">
body.chakra-ui-light > div.chakra-portal > ul#chakra-toast-manager-bottom-right
<ul role="region" aria-live="polite" id="chakra-toast-manager-bottom-right" style="position: fixed; z-index: var(--toast-z-index, 5500);">

hamirmahal commented 1 month ago

Additional context

https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1716840291297-34295.report.html may provide some more context.

straker commented 1 month ago

Thanks for the issue. It looks like the docs for aria-allowed-role is wrong and is a copy paste of aria-roles. The docs should say that the ul element is not allowed to have a role of region in accordance with ARIA in HTML.

Also it looks like the Lighthouse summary of the rule is incorrect as well as that description looks like it fits aria-roles instead of aria-allowed-role. You might want to open an issue with Lighthouse and bring that up.

hamirmahal commented 1 month ago

You're welcome. I opened https://github.com/GoogleChrome/lighthouse/issues/16023 in Lighthouse. Thanks for the details.

adamraine commented 1 month ago

The docs should say that the ul element is not allowed to have a role of region in accordance with ARIA in HTML.

@straker Is this a complete description of the rule (i.e. it applies only to ul elements)? I'm wondering how we should be phrasing the Lighthouse audit description if we can't base it on https://dequeuniversity.com/rules/axe/4.9/aria-allowed-role (until the docs are updated)

christophe-g commented 4 weeks ago

I am landing here from lighthouse where I am getting a Values assigned to role="" are not valid ARIA roles. error for role="toolbar", which is a valid aria role, but not listed on the axe doc allowed-role.

Not creating a new issue as I think it is related to this one.

straker commented 3 weeks ago

@adamraine We're working on updating the docs. In the meantime this is the description and why important sections of the rule we're going with

Rule Description

Not all WAI-ARIA role values are allowed to be used on every HTML element. Many HTML elements can only be assigned certain WAI-ARIA roles.

Why Important

Using WAI-ARIA roles where they are not allowed can interfere with the accessibility of the web page. Using an invalid HTML element and ARIA role combination will, at best, result in no effect on the accessibility of the application and, at worst, may trigger behavior that disables accessibility for entire portions of an application.

When ARIA roles are used on HTML elements that are not in accordance with the HTML in ARIA specification, they conflict with the semantics of the elements which can cause assistive technology products report nonsensical user interface (UI) information that does not represent the actual UI of the document.

adamraine commented 3 weeks ago

Not all WAI-ARIA role values are allowed to be used on every HTML element. Many HTML elements can only be assigned certain WAI-ARIA roles.

Sorry still a bit confused, are these two statements saying the same thing? Trying to write the title of the Lighthouse audit and I'm not sure if I need to somehow combine these two statements in order to summarize the rule, or if the title could just be "Uses ARIA roles on incompatible elements" (or something).

straker commented 3 weeks ago

The updated docs for the rule are now available on Deque university https://dequeuniversity.com/rules/axe/4.9/aria-allowed-role

dequelabs / axe-core