search

pods-framework / pods

The Pods Framework is a Content Development Framework for WordPress - It lets you create and extend content types that can be used for any project. Add fields of various types we've built in, or add your own with custom inputs, you have total control.
https://pods.io/
GNU General Public License v2.0
1.06k stars 264 forks source link

Accessibility test report using Cypress and cypress-axe #7201

Open heybran opened 9 months ago

heybran commented 9 months ago

Problem to Solve

Hey @sc0ttkclark

I want to automate my accessibility testing on Pods (moving on from manual testing that I did before today) with Cypress and cypress-axe. And, I have created a repo for me to store testing files, a single test file looks like this: https://github.com/heybran/pods-a11y-tests/blob/main/cypress/e2e/edit-pod-screen/add-new-group.cy.js

describe('Edit pod screen', () => {
  beforeEach(() => {
    cy.visit(Cypress.env('host') + '/wp-admin/');
    cy.login(Cypress.env('username'), Cypress.env('password'));
  });

  it('Edit pod screen - add new group - has no detectable a11y violations on load (custom configuration)', () => {
    cy.visit(Cypress.env('host') + '/wp-admin/admin.php?page=pods&action=edit&id=6')
      .injectAxe()
      /**
       * cy.click() can only be called on a single element. Your subject contained 2 elements. 
       * Pass { multiple: true } if you want to serially click each element.
       * @see https://docs.cypress.io/api/commands/click
       */
      .get('.pods-button-group_container:first-child [aria-label="Add new field group to the Pod"]').click()
      // Add a new group modal - default wp modal component.
      .checkA11y('.components-modal__screen-overlay');
  });

  it('Edit pod screen - add new group - advanced tab - has no detectable a11y violations on load (custom configuration)', () => {
    cy.visit(Cypress.env('host') + '/wp-admin/admin.php?page=pods&action=edit&id=6')
      .injectAxe()
      /**
       * cy.click() can only be called on a single element. Your subject contained 2 elements. 
       * Pass { multiple: true } if you want to serially click each element.
       * @see https://docs.cypress.io/api/commands/click
       */
      .get('.pods-button-group_container:first-child [aria-label="Add new field group to the Pod"]').click()
      .get('[aria-controls="advanced-tab"]').click()
      // Add a new group modal - default wp modal component.
      .checkA11y('.components-modal__content');
  });
});

So I need to create an new issue to share the accessibility reports from Cypress and then move on to fix those issues.

Manage pods screen

1. Color contract

Command:      a11y error!
Id:           color-contrast
Impact:       serious
Tags:         Array(8)
Description:  Ensures the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds
Help:         Elements must meet minimum color contrast ratio thresholds
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/color-contrast?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element has insufficient color contrast of 1.96 (foreground color: #f6f6e8, background color: #95bf3b, font size: 9.8pt (13px), font weight: normal). Expected contrast ratio of 4.5:1"

html: 
"<a href=\"https://friends.pods.io/donations/become-a-friend/?utm_source=pods_plugin_callout&amp;utm_medium=link&amp;utm_campaign=friends_2023_docs\" target=\"_blank\" rel=\"noreferrer\" class=\"pods-admin_friends-callout_button--join\">\n\t\t\t\tDonate Now »\n\t\t\t</a>"

2. Link accessible name

Command:      a11y error!
Id:           link-name
Impact:       serious
Tags:         (12) ['cat.name-role-value', 'wcag2a', 'wcag244', 'wcag412', 'section508', 'section508.22.a', 'TTv5', 'TT6.a', 'EN-301-549', 'EN-9.2.4.4', 'EN-9.4.1.2', 'ACT']
Description:  Ensures links have discernible text
Help:         Links must have discernible text
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/link-name?application=axeAPI

failureSummary: 
"Fix all of the following:\n  Element is in tab order and does not have accessible text\n\nFix any of the following:\n  Element does not have text that is visible to screen readers\n  aria-label attribute does not exist or is empty\n  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n  Element has no title attribute"

html: 
"<a href=\"/wp-admin/admin.php?page=pods&amp;pods_callout_dismiss=friends_2023_docs\" class=\"pods-admin_friends-callout_close\">"
impact

Edit pod screen

1. Accessible name

Command:      a11y error!
Id:           aria-command-name
Impact:       serious
Tags:         Array(8)
Description:  Ensures every ARIA button, link and menuitem has an accessible name
Help:         ARIA commands must have an accessible name
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/aria-command-name?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element does not have text that is visible to screen readers\n  aria-label attribute does not exist or is empty\n  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n  Element has no title attribute"

html: 
"<div class=\"pods-field-group_handle\" role=\"button\" tabindex=\"0\" aria-disabled=\"false\" aria-roledescription=\"sortable\" aria-describedby=\"DndDescribedBy-0\"><span class=\"dashicon dashicons dashicons-menu\"></span></div>"

2. Color contract

Command:      a11y error!
Id:           color-contrast
Impact:       serious
Tags:         (8) ['cat.color', 'wcag2aa', 'wcag143', 'TTv5', 'TT13.c', 'EN-301-549', 'EN-9.1.4.3', 'ACT']
index-21ca8685.js:133730 Description:  Ensures the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds
Help:         Elements must meet minimum color contrast ratio thresholds
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/color-contrast?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element has insufficient color contrast of 4.04 (foreground color: #007cba, background color: #f1f1f1, font size: 9.8pt (13px), font weight: normal). Expected contrast ratio of 4.5:1"

html: 
"<button type=\"button\" aria-label=\"Edit the slug\" class=\"components-button is-secondary\">Edit</button>"

3. nested-interactive

Command:      a11y error!
Id:           nested-interactive
Impact:       serious
Tags:         (7) ['cat.keyboard', 'wcag2a', 'wcag412', 'TTv5', 'TT6.a', 'EN-301-549', 'EN-9.4.1.2']
Description:  Ensures interactive controls are not nested as they are not always announced by screen readers or can cause focus problems for assistive technologies
Help:         Interactive controls must not be nested
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/nested-interactive?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element has focusable descendants"

html: 
"<div tabindex=\"0\" role=\"button\" class=\"pods-field-group_title\" aria-label=\"Press and hold to drag this item to a new position in the list\">"

Edit pod screen - add new group

1. Accessible name

Command:      a11y error!
Id:           aria-command-name
Impact:       serious
Tags:         Array(8)
Description:  Ensures every ARIA button, link and menuitem has an accessible name
Help:         ARIA commands must have an accessible name
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/aria-command-name?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element does not have text that is visible to screen readers\n  aria-label attribute does not exist or is empty\n  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n  Element has no title attribute"

html: 
"<span tabindex=\"0\" role=\"button\" class=\"pods-help-tooltip__icon\" aria-expanded=\"false\"><span class=\"dashicon dashicons dashicons-editor-help\"></span></span>"

failureSummary: 
"Fix any of the following:\n  Element does not have text that is visible to screen readers\n  aria-label attribute does not exist or is empty\n  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n  Element has no title attribute"

html: 
"<span tabindex=\"0\" role=\"button\" class=\"pods-help-tooltip__icon\" aria-expanded=\"false\"><span class=\"dashicon dashicons dashicons-editor-help\"></span></span>"

2. aria-required-children

Command:      a11y error!
Id:           aria-required-children
Impact:       critical
Tags:         (5) ['cat.aria', 'wcag2a', 'wcag131', 'EN-301-549', 'EN-9.1.3.1']
Description:  Ensures elements with an ARIA role that require child roles contain them
Help:         Certain ARIA roles must contain particular children
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/aria-required-children?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Element has children which are not allowed: [role=button]\n  Element uses aria-busy=\"true\" while showing a loader"
html: 
"<div class=\"pods-settings-modal__tabs\" role=\"tablist\" aria-label=\"Pods Field Group Settings\">"

3. Ensures all ARIA attributes have valid values

Command:      a11y error!
Id:           aria-valid-attr-value
Impact:       critical
Tags:         (5) ['cat.aria', 'wcag2a', 'wcag412', 'EN-301-549', 'EN-9.4.1.2']
Description:  Ensures all ARIA attributes have valid values
Help:         ARIA attributes must conform to valid values
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/aria-valid-attr-value?application=axeAPI

failureSummary: 
"Fix all of the following:\n  Invalid ARIA attribute value: aria-controls=\"basic-tab\""

html: 
"<div class=\"pods-settings-modal__tab-item pods-settings-modal__tab-item--active\" aria-controls=\"basic-tab\" role=\"button\" tabindex=\"0\">Group Details</div>"

failureSummary: 
"Fix all of the following:\n  Invalid ARIA attribute value: aria-controls=\"advanced-tab\""

html: 
"<div class=\"pods-settings-modal__tab-item\" aria-controls=\"advanced-tab\" role=\"button\" tabindex=\"0\">Advanced</div>"

4. Color contract

Command:      a11y error!
Id:           color-contrast
Impact:       serious
Tags:         (8) ['cat.color', 'wcag2aa', 'wcag143', 'TTv5', 'TT13.c', 'EN-301-549', 'EN-9.1.4.3', 'ACT']
Description:  Ensures the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds
Help:         Elements must meet minimum color contrast ratio thresholds
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/color-contrast?application=axeAPI

TODO: Need to find all 10 elements that have color contract issue.

5. Heading order

Command:      a11y error!
Id:           heading-order
Impact:       moderate
Tags:         (2) ['cat.semantics', 'best-practice']
Description:  Ensures the order of headings is semantically correct
Help:         Heading levels should only increase by one
Helpurl:      https://dequeuniversity.com/rules/axe/4.8/heading-order?application=axeAPI

failureSummary: 
"Fix any of the following:\n  Heading order invalid"

html: 
"<h3 class=\"pods-form-ui-heading pods-form-ui-heading-visibility\" id=\"heading-visibility\">Visibility</h3>"

TODO: Will have more reports once more testing specs are run

Proposed Solution

I will go through them one by one and fix them.

sc0ttkclark commented 9 months ago

Would it be helpful to add this kind of code directly to the repo for you to more easily use them on Pods itself?

heybran commented 9 months ago

At the moment, it's easier for me to do it on a separate local repo, but it would make sense to move these test specs into Pods code base in future.