mfrachet / cypress-audit

⚡ Run Lighthouse and Pa11y audits directly in your E2E test suites
https://mfrachet.github.io/cypress-audit/
MIT License
351 stars 44 forks source link

feat(kayle): improve performance over 10,000% and coverage accuracy #222

Closed j-mendez closed 1 year ago

j-mendez commented 1 year ago

Whats Changed

Improve performance of accessibility testing using the fastest and accurate accessibility runner. The runner has over 10,000% increase on performance on average web test cases and is drastically more accurate reducing false positives. There are also custom runner forks used that fix bugs found in axe and htmlcs at more acceptable rate of the evolution of standards and changes to the web.

Extra:

You can also add different runners like htmlcs to extend the results. All runners are forks of the main libs with vast testing across diverse web pages and use cases.

    const results = await kayleLib({ browser, runners: ["axe", "htmlcs"], ...opts, origin: url });

Kayle is a battle tested library that receives constant updates and is porting the core to Rust with native platform targeting C++ extensions and wasm builds allowing for development safety and portability.

Pa11y is slow and should not be used as it misses out on a lot of coverage. Also our axe fork covers more than the core in folds.

j-mendez commented 1 year ago

We had major benchmarks and test in earlier commits on https://github.com/a11ywatch/kayle. We removed them since we saw the rate of progress we were moving at compared to the others. You can check out an earlier commit to see the increased coverage as well and run the test. There is also many features that come into kayle to allow testing without being blocked like using brave-adblock for adblocking, flexible and performant network intercepting, and ability to test raw HTML and lint html without a browser. There is also a function called autoKayle that can run parallel across the entire website. Not as good as using the concurrent setup with the a11ywatch suite but, it works enough for small-medium websites. When you have over 5k pages like HBO.com and want things done within a minute that is when A11yWatch comes into play.