search

EqualifyEverything / equalify

A web accessibility platform, managing issues by integrating with A11Y services.
https://equalify.app
Other
117 stars 20 forks source link

Automated AI (GPT-Powered) WCAG Scanning #220

Closed bbertucc closed 1 year ago

bbertucc commented 1 year ago

Guiding Questions

Can AI tools, specifically Generative Pre-training Transformers (GPTs), find WCAG errors better than existing automated scans? Where are the benefits? Where are the shortcomings?

Hypothesis

Some standards, specifically qualitative standards, may be able to be assessed via a GPT tool with more accuracy than any existing automated tool. These GPT tools will also be able to assess quantitative standards, but those testing those standards would not be worth the processing cost of using a GPT tool versus a simple automated tool, like axe-core, to test. Therefore, GPT-based WCAG scanning will be best served for specific use cases to enhance the work of a skilled accessibility expert.

Testing Process

I've outlined the testing process in the article a11yGPT: Experimental AI-based Accessibility Testing. As stated there, all updates will be reported on this ticket in addition to the #a11y-ai channel of a11y Slack.

Ultimate Goal

My ultimate goal is to understand what WCAG standards GPT tools can reliably scan source code for and incorporate that scan into the open source a11y platform I created, https://equalify.app.

Join In!

I should note that I am currently looking for sponsors and partners to help with this project.

If you are interested in helping or have any questions at all, please contact Decubing.

bbertucc commented 1 year ago

After some testing it seems that GPT tools just aren't ready for testing sites yet. They report too many false positives. AI seems largely outside Equalify's scope - for now.