search

allure-framework / allure2

Allure Report is a flexible, lightweight multi-language test reporting tool. It provides clear graphical reports and allows everyone involved in the development process to extract the maximum of information from the everyday testing process
https://allurereport.org/
Apache License 2.0
3.93k stars 695 forks source link

Allure report accessibility issues #2264

Open cmorten opened 6 months ago

cmorten commented 6 months ago

Describe the Bug

There are a number of areas of the generated Allure report that are not considered accessible to all users under the Web Content Accessibility Guidelines (WCAG) 2. This is currently impacting a range of users such as those with poor vision, colour-blindness, folks who make use of assistive technologies such as screen readers, etc.

It would be fantastic to start addressing some of these issues so Allure reports can be easily used by everyone regardless of their personal access requirements.

As a starter, here is a non-exhaustive list of serious/critical issues broken down by top level view, captured using the Axe DevTools Chrome Extension:

(I've not repeated cross-cutting issues for each page).

Steps to Reproduce

  1. Generate Allure report
  2. View Allure report in a browser
  3. Observe visual, keyboard, and assistive technology issues

For reproducing the issues reported above:

  1. Open Chrome browser
  2. Install Axe DevTools extension from https://chromewebstore.google.com/detail/axe-devtools-web-accessib/lhdoppojpmngadmnindnejefpokejbdd
  3. Open an Allure report in the Chrome browser
  4. In Chrome DevTools navigate to the "axe DevTools" tab
  5. Click "Scan ALL of my page"
  6. Observe the reported results for that page

The Chrome Developer docs also have a number of runthroughs and resources for observing / testing accessibility issues, see https://developer.chrome.com/docs/devtools/accessibility/reference.

Expected Behaviour

There should be no critical/serious accessibility issues with Allure reports, allowing all users to successfully understand a report regardless of their personal access requirements.

Screenshots or Additional Context

No response

What Language are you using?

JavaScript, TypeScript

What Framework/Allure Integration you are using?

allure-playwright

What version of Allure Integration you are using?

2.10.0

What version of Allure Report you are using?

2.25.0

Code of Conduct

noomorph commented 4 months ago

According to an experienced NVDA user (who is totally blind), he was able to navigate the report without any issues. Although, it would be great if this bug report could be fixed in the future.

Zdråvej, trěboval jesm 2 razy obnoviti strånicų F5, bo najpŕvo ne jesm viděl ničto, ale potom je bylo vśe dobro. Hello, I had to refresh the F5 page twice, because at first I didn't see anything, but then everything was fine.

I'm not sure why he needed to refresh twice, but it could be due to the hosting server issues – I can't pinpoint the issue at the moment.

cmorten commented 4 months ago

@noomorph great stuff testing with a real AT user!

I would say that sounds consistent with the Axe analysis - the only recommendations that could be impacting a screen reader are the search input label and the timeline links (there may be other bits that Axe hasn’t found as well). If NVDA is handling it well then awesome, probably worth still doing as best practice, and may still be beneficial for other screen readers such as JAWS and VoiceOver.

The majority of issues according to the report are for colour contrast which won’t impact no vision folks who rely on a screen reader, but will impact folks with low vision or vision deficiencies such as colour blindness who are navigating the report visually.

baev commented 4 months ago

By the way, we have this ticket on the radar, although it isn't yet prioritised.

@cmorten, is it a blocker for you? Do you have some team members who are struggling with the Allure Report at the moment, or do you need to for compliance reasons?

cmorten commented 4 months ago

No blocker for me, so no rush on my end!

I have a project for which I have been evaluating different reporter options, and tend to run a quick scan as a matter of course to check accessibility as part of evaluating options.