microsoft / axe-windows

Automated accessibility testing engine for Windows applications
MIT License
136 stars 62 forks source link

feat(async-api): Multiple scan roots are always enabled #763

Closed DaveTryon closed 2 years ago

DaveTryon commented 2 years ago

Details

This is a BREAKING change to the CLI. Scripts that called the CLI with the --aremultiplescanrootsenabled parameter will reject the arguments. To be fair, though, we know of nobody who uses this parameter. The only known case of scanning multiple top-level windows goes directly to the Automation API and does not use the CLI.

When we added multiple scan roots, we jumped through some hoops to ensure it would be non-breaking. The 2.0 Automation API is a breaking change, so we've decided to simplify multiple scan roots. They're always enabled and currently have no means of disabling them. If requested, it's easy to add at a later time.

One other change in behavior to call out: For anyone who used the Automation API and specified multiple root window support, scans would generate an a11ytestfile even if only one top-level window was scanned and no errors were found. This changes this. When a single top-level window is scanned, the a11ytest file will be generated only if errors exist. When multiple top-level windows are scanned, an a11ytest file will be generated per top-level window, regardless of error status.

Motivation
Context

There is no way to say "Only scan one top-level window if multiple top-level windows exist in the process". This is by design, as the order in which top-level windows is deterministic but hard to concisely describe. We punted support of this before, and are not adding it here.

Pull request checklist

codecov-commenter commented 2 years ago

Codecov Report

Merging #763 (a39f3ac) into main (4fc19c3) will increase coverage by 0.87%. The diff coverage is 79.15%.

@@            Coverage Diff             @@
##             main     #763      +/-   ##
==========================================
+ Coverage   73.70%   74.57%   +0.87%     
==========================================
  Files         398      420      +22     
  Lines       12046    12895     +849     
==========================================
+ Hits         8878     9616     +738     
- Misses       3168     3279     +111     
Impacted Files Coverage Δ
src/Actions/Actions/ControlPatternAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/CustomUIAAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/ListenAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/LoadActionParts.cs 90.90% <ø> (ø)
src/Actions/Actions/SetDataAction.cs 0.00% <0.00%> (ø)
...rc/Actions/Attributes/InteractionLevelAttribute.cs 0.00% <ø> (ø)
src/Actions/Contexts/ElementDataContext.cs 86.36% <ø> (ø)
src/Actions/Misc/ExtensionMethods.cs 7.46% <0.00%> (-31.60%) :arrow_down:
src/Actions/Resources/ErrorMessages.Designer.cs 0.00% <ø> (-55.56%) :arrow_down:
src/Actions/Trackers/BaseTracker.cs 51.02% <0.00%> (-16.55%) :arrow_down:
... and 230 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.