search

dequelabs / axe-core

Accessibility engine for automated Web UI testing
https://www.deque.com/axe/
Mozilla Public License 2.0
6.03k stars 784 forks source link

What is the correct percentage of issues that axe actually finds it when running? #4415

Open djalmaaraujo opened 7 months ago

djalmaaraujo commented 7 months ago

Product

axe Extension

Question

Hello! Could you please clarify the correct percentage that axe-core usually gets when running?

I ask this because sometimes we need to prepare presentations and slides with this information. Yesterday, I was looking at the Deque DevTools website, and it says it gets 80% of accessibility issues. Still, if you come here in the axe-core repository, you have this quote:

With axe-core, you can find on average 57% of WCAG issues automatically. Additionally, axe-core will return elements as "incomplete" where axe-core could not be certain, and manual review is needed.

The website statement for DevTools:

image

Axe is amazing, but in my position, I have to report to some other leadership asking what's the impact of using tools for A11Y, and it would be amazing to have some clarification on this subject.

Thank you for all your work!

straker commented 7 months ago

Thanks for the question. So there are two different things that determine those numbers. First automated testing frameworks such as axe-core can only catch about 57% of accessibility issues. The 80% comes from the axe DevTools extension which can use our Intelligent Guided Tests (IGT) to help increased that 57% to 80%. So if you are able to use IGT it can help catch up to 80% issues, otherwise its about 57% coverage.