Closed mockdeep closed 8 months ago
Hey @mockdeep just following up to see if this is still an issue.
@michael-siek I think this was probably due to the page not being fully loaded. We ended up implementing some retry logic that caught the error and retried with a backoff. This worked pretty well for a while, but we had this other issue with flakiness in version 4.8.0.
The workaround in our code was to ensure that the page was fully loaded with an extra assertion before doing a11y checks. For what it's worth, it seems like ideally axe-core-rspec
would handle the possibility of flake related to page loading, similar to the way that Capybara does. Capybara provides a synchronize
method where you can catch certain errors and have it retry until the timeout.
I'm fine closing this out, though, if you don't think it's appropriate.
@mockdeep thank you for the response! will close and can reopen if needed.
As gems
being able to scan the page successfully and nothing is reproducing .
https://github.com/dequelabs/axe-core-gems/assets/47409579/37aac8d8-8556-408a-bc6b-03183b65b881
Label | Value |
---|---|
Product | axe-core-gems/rspec-cucumber |
Version | 4.8.2 |
OS-Details | MAC- Intel Core i7 - 11.6.8 && Windows 11 |
BrowserDetails | Chrome Version 121.0.6167.139 && Firefox 122.0.1 |
We see occasional flake in our builds where
axe
isn't loaded into the page for some reason. Maybe it isn't ready, yet?Stack Trace
``` Minitest::UnexpectedError: UncaughtThrowError: uncaught throw {"errorMessage"=>"window.axe is undefined", "passes"=>[], "timestamp"=>"Wed May 04 2022 15:41:38 GMT-0700 (Pacific Daylight Time)", "url"=>"", "violations"=>[]} /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/api/run.rb:120:in `throw' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/api/run.rb:120:in `run_partial_recursive' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/api/run.rb:36:in `analyze_post_43x' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/core.rb:18:in `call' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/matchers/be_axe_clean.rb:22:in `audit' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/axe-core-api-4.4.0/lib/axe/matchers/be_axe_clean.rb:26:in `matches?' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/rspec-expectations-3.11.0/lib/rspec/expectations/handler.rb:51:in `block in handle_matcher' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/rspec-expectations-3.11.0/lib/rspec/expectations/handler.rb:27:in `with_matcher' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/rspec-expectations-3.11.0/lib/rspec/expectations/handler.rb:48:in `handle_matcher' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/rspec-expectations-3.11.0/lib/rspec/expectations/expectation_target.rb:65:in `to' /home/circleci/app/vendor/bundle/ruby/3.0.0/gems/rspec-expectations-3.11.0/lib/rspec/expectations/expectation_target.rb:101:in `to' /home/circleci/app/spec/support/accessibility_overrides.rb:19:in `click_button' /home/circleci/app/spec/features/doc_setup/auto_detect_fields_spec.rb:72:in `block (3 levels) in