microsoft / axe-windows

Automated accessibility testing engine for Windows applications
MIT License
136 stars 62 forks source link

fix: Reduce chromium noise #951

Closed DaveTryon closed 1 year ago

DaveTryon commented 1 year ago

Details

As called out in #836, the ChromiumContentShouldUseWebScanner has caused a lot of confusion. This PR aims to reduce that confusion. It does the following:

Motivation

Address #836, make the ChromiumContentShouldUseWebScanner rule less intrusive

Attachments

BeforeAndAfter.zip contains 2 a11ytest files

Here are how the numbers line up

Message Count before Count after Count of Chromium elements
An element must not have the same Name and LocalizedControlType as its parent 7 0 7
An element of the given ControlType must not support the Invoke pattern 1 1 0
An element of the given ControlType must support the Text pattern 1 0 1
An on-screen element must not have a null BoundingRectangle property 5 5 0
Chromium components should be scanned with a web-based scanner 63 2 63
The Name must not include the same text as the LocalizedControlType 2 2 0
The Name property must not include the element's control type 2 2 0

Things to note on this table:

If you open After.a11ytest and inspect the children of the Chromium documents, the UIA properties are reported but no tests were run. That's exactly what this change was intended to do.

Screenshots

Here are visual representations of the test results with all test results displaying. Notice the difference in the HTML document space:

File Snapshot (click to enlarge) Test Results (click to enlarge)
Before image image
After image image
Context

Pull request checklist

codecov-commenter commented 1 year ago

Codecov Report

Merging #951 (737df70) into main (4fc19c3) will increase coverage by 1.12%. Report is 331 commits behind head on main. The diff coverage is 79.02%.

:exclamation: Current head 737df70 differs from pull request most recent head e366517. Consider uploading reports for the commit e366517 to get more accurate results

@@            Coverage Diff             @@
##             main     #951      +/-   ##
==========================================
+ Coverage   73.70%   74.82%   +1.12%     
==========================================
  Files         398      423      +25     
  Lines       12046    13093    +1047     
==========================================
+ Hits         8878     9797     +919     
- Misses       3168     3296     +128     
Files Changed Coverage Δ
src/Actions/Actions/ControlPatternAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/CustomUIAAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/ListenAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/PrivacyExtensions.cs 100.00% <ø> (ø)
src/Actions/Actions/SetDataAction.cs 0.00% <0.00%> (ø)
...rc/Actions/Attributes/InteractionLevelAttribute.cs 0.00% <0.00%> (ø)
src/Actions/Misc/ExtensionMethods.cs 7.46% <0.00%> (-31.60%) :arrow_down:
src/Actions/Resources/ErrorMessages.Designer.cs 0.00% <ø> (-55.56%) :arrow_down:
src/Automation/AxeWindowsAutomationException.cs 33.33% <ø> (-33.34%) :arrow_down:
src/Automation/Data/OutputFile.cs 88.88% <ø> (ø)
... and 144 more

... and 241 files with indirect coverage changes