search

paypal / AATT

Automated Accessibility Testing Tool
BSD 3-Clause "New" or "Revised" License
602 stars 106 forks source link

UI shows 10 issues whereas API returns only 2 #34

Closed mustufa84 closed 3 years ago

mustufa84 commented 6 years ago

I am trying to research on this tool and I am facing an issue where running the htmlcs through the UI shows me 7 errors but when I use the API and pass the same html code, it only shows me 2 errors

below is my html




What am I doing wrong?

mpnkhan commented 6 years ago

Could you give details of how you are testing? Are you using it part of Automation test case? or are you testing it with UI -> TestHTML? Note that testhtml will default to WCAG2AA and default will return notices and warnings.

You can also test the api with http://localhost:3000/test/PAETAPI.html.

Now nemo accessibility plugin is independent of AATT, so you can test easily with nemo accessibility plugin as well.

jaredlevi commented 6 years ago

I'm also having this issue, the UI returns 24 errors while the API is only returning 2.

mpnkhan commented 6 years ago

The results are because of the errors/ warning/notices returned in UI. The default engine used etc. API returns only error by default. Kindly let us know what's the engine you use in UI and ruleset selected.

andrew041 commented 6 years ago

I am seeing this issue as well. From as far as I could tell, all of the settings were the same. Using htmlcs, showing errors, warnings, and notices (1,2,3 in the api), output type of json, and I was hitting the same website both with the UI and API. I tried axe and chrome with the API and only got 'undefined' as a response.

mpnkhan commented 5 years ago

This should have been fixed with https://github.com/paypal/AATT/pull/50. Not AATT should return error, warning notice etc. . Kindly re-check