search

IBMa / equal-access

IBM Equal Access Accessibility Checker contains tools to automate accessibility checking from a browser or in a continuous development/build environment
https://www.ibm.com/able/toolkit/tools#develop
Apache License 2.0
629 stars 81 forks source link

[BUG]: Differences between Chrome and Firefox #2028

Open tombrunet opened 1 week ago

tombrunet commented 1 week ago

Project

accessibility-checker-engine

Browser

Firefox

Operating system

MacOS

Description

When building the Java tool, I tried with the Firefox Selenium engine. These testcases don't match the results. It might be because Firefox is auto-fixing some things, but we should at least verify:

"aria_role_redundant_ruleunit", "Fail.html"
"style_focus_visible_ruleunit", "CSS-used.html"
"object_text_exists_ruleunit", "act_fail_3.html"
"aria_attribute_conflict_ruleunit", "aria-hidden.html"
"aria_attribute_valid_ruleunit", "ValidRoleSpecified.html"
"aria_attribute_valid_ruleunit", "ValidAttribute.html"
"aria_attribute_valid_ruleunit", "ValidRoleSpecifiedValidAttribute.html"
"aria_attribute_valid_ruleunit", "InvalidAttribute.html"
"aria_attribute_valid_ruleunit", "InValidRoleInvalidAttribute.html"
"aria_attribute_valid_ruleunit", "InValidRoleSpecified.html"
"aria_attribute_valid_ruleunit", "ValidRoleSpecifiedInvalidAttribute.html"
"aria_attribute_valid_ruleunit", "ValidRoleSpecifiedMultiple.html"
"aria_attribute_valid_ruleunit", "InValidRoleSpecifiedMultiple.html"
"aria_attribute_valid_ruleunit", "area_element_test.html"
"aria_attribute_valid_ruleunit", "elementsWithSupportingAttributes.html"
"aria_role_valid_ruleunit", "area_no_href.html"
"element_scrollable_tabbable_ruleunit", "act_fail2.html"
"element_scrollable_tabbable_ruleunit", "textarea_pass2.html"
"element_scrollable_tabbable_ruleunit", "act_fail1.html"
"fieldset_label_valid_ruleunit", "FieldSet-hasarialabel.html"
"fieldset_label_valid_ruleunit", "FieldSet-nested.html"
"fieldset_label_valid_ruleunit", "test_mixed_1.html"
"a_text_purpose_ruleunit", "A-nonTabable.html"
"a_text_purpose_ruleunit", "A-slot-text-error2.html"
"a_text_purpose_ruleunit", "A-slot-text-error1.html"
"target_spacing_sufficient_ruleunit", "link_in_text.html"
"target_spacing_sufficient_ruleunit", "block_element_inline.html"
"aria_landmark_name_unique_ruleunit", "example_0_fail.html"

Steps to reproduce

Will be easier once the java checker is merged. From the java-accessibility-checker directory, can run:

mvn test -Dtest="AccessibilityCheckerSeleniumFFTest"

Will have to comment out the skip items to see them fail.

shunguoy commented 1 week ago

Chrome and FF with different results: http://127.0.0.1:5500/test/v2/checker/accessibility/rules/aria_role_redundant_ruleunit/Fail.html potential causes: (1) the browser removes/fixes some attributes; (2) some functions may be different; (3) potential rule issue.

shunguoy commented 1 week ago

Investigate the root issues, then open bugs on each finding.