search

ttsukagoshi / axe-scan

A CLI tool to test web accessibility on multiple web pages based on a list of URLs in a text file. Run on Node.js and uses axe-core and Puppeteer as its testing and browsing engines.
https://www.scriptable-assets.page/axe-scan/
MIT License
25 stars 5 forks source link
a11y axe-core cli nodejs npm-package puppeteer web-accessibility-tool

readme

axe-scan: Testing web accessibility of multiple web pages

Accessibility-alt-text-bot CodeQL Lint Code Base npm version code style: prettier

A CLI tool to test web accessibility on multiple web pages based on a list of URLs in a text file. Run on Node.js and uses axe-core and Puppeteer as testing and browsing engines.

Table of Contents

The basic idea

Making a website accessible is not just about social equity and legal compliance; accessibility is fundamental to developing and maintaining a user-oriented website. Improved user experience is likely to be favored by search engines like Google, which should be a heads-up for the management-level members of the organization who are less interested in this topic.

Web accessibility is often brought up in the context of the responsibilities of designers and developers, and indeed, the Web Content Accessibility Guidelines (WCAG) state itself that it is intended for:

  • Web content developers (page authors, site designers, etc.)
  • Web authoring tool developers
  • Web accessibility evaluation tool developers
  • Others who want or need a standard for web accessibility, including for mobile accessibility

(Who WCAG is for from WCAG 2.0 Overview)

Looking a little deeper into the actual contents of the WCAG, however, you will see that EVERY member of those related to developing and maintaining a website, not just the developers and engineers, need to be aware of the requirements to achieve a certain level of accessibility and keep that level. Axe-scan aims to provide a common ground of discussion to improve the website's accessibility between tech professionals and other members who are less familiar with WCAG through the following features:

Testing web accessibility: the basic flow

This section lays out the basic outlines of testing web accessibility using axe-scan. See the later sections for details.

1. Install axe-scan

See the Installation section.

2. Setup axe-scan and set locale

Setup axe-scan in your working directory:

axe-scan init

If you want to change the language of the axe-scan output:

axe-scan config --change-value locale=ja

The above example sets the locale to ja or Japanese. See the locale in Configuring axe-scan section for more details on the values that it can take.

3. Create urls.txt

In your current working directory, create a list of the URLs on which to conduct the accessibility test. The URLs should be separated by line breaks. Name the file urls.txt. See Run the scan section for more detail.

4. Initial accessibility test

Conduct the initial test and save the results to results.csv:

axe-scan run > results.csv

A list of around 1,000 pages will take approx. 1 hour to complete.

5. Review the results, resolve issues, and create an allowlist

Open the results.csv and check the contents. You will likely see that under the Result Types column, there are violations and incomplete items.

Violations are elements that clearly violate one or more of the WCAG Success Criteria. Address these issues. Note that there may be multiple rows for a single issue if it has more than one way to resolve it. Once handled properly using one of the methods, the set of items will all disappear.

Incomplete are elements that axe-core could not clearly determine whether it is a violation or not. Check the item manually and determine if it is a violation or not. If no problems were found, flag the row to include it in the allowlist for future use.

Create an allowlist.csv by filtering the results.csv for items that you marked so. Include the first row, i.e., the header row, when you save allowlist.csv as a new file.

6. Accessibility testing: second run

Execute the second run on the accessibility test, this time taking into account the contents of the allowlist.csv:

axe-scan run --allowlist allowlist.csv > results-allowlisted.csv

Check the created CSV file results-allowlisted.csv and see that items in the allowlist.csv are not present. In fact, if you have resolved the violations in the previous step, this CSV file should be empty.

7. Create a summary report

To create a summary report based on the WCAG Success Criteria, run:

axe-scan summary --allowlist allowlist.csv > summary-allowlisted.csv

After the previous steps, all items should be PASS or INAPPLICABLE.

8. Share the files with your team

You can use the allowlist.csv for periodical web accessibility tests using axe-scan to easily detect new violations and incomplete items.

9. Use with manual checks

Axe-scan is not a one-size-fits-all accessibility test. There are some WCAG Success Criteria (SC) that axe-scan cannot check, so to create a full checklist of all the WCAG SC, you will have to use axe-scan together with other means like manual reviews or tests using other specialized tools.

Installation

Prerequisite: Node.js

To install axe-scan, you must have Node.js installed as a prerequisite. To check, enter on your Terminal, Powershell, Command Prompt, or your preferred client:

node -v

If this returns an error, install the latest LTS version of Node.js before you proceed below. This document will not go into the details, but there are ways to make version management of Node.js easier for macOS and Linux users:

Install axe-scan

On your Terminal, enter:

npm install -g axe-scan

This installs axe-scan globally, i.e., all executions run on your account will use the same version of axe-scan.

If you want to separate the versions depending on your working directory, install axe-scan locally without the -g option:

npm install axe-scan

If you want to install a specific version of axe-scan, add the version number to the package name:

npm install axe-scan@1.0.0

Update Package

To update your installed axe-scan:

npm update -g axe-scan

Remove the -g option like in the Install axe-scan section, depending on your needs.

Usage

Initiate

axe-scan init [options]

Create a axe-scan.config.json in the working directory with the default values. If there is an existing config file in the directory, it will ask you whether to update the file or to abort. If you have placed a config file at your home directory ~/axe-scan.config.json, axe-scan init will copy that setting instead of the default values.

Use the --global option to create the config file in your home directory rather than your current working directory:

axe-scan init --global

Options

Default settings

{
  "axeCoreTags": ["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"],
  "resultTypes": ["incomplete", "violations"],
  "filePath": "./urls.txt",
  "locale": "en"
}

See Configuring axe-scan section for more detail. User settings will be determined in the following order: (1) ./axe-scan.config.json file in the working directory, (2) ~/axe-scan.config.json file in the user's home directory, and (3) default settings.

Configure

axe-scan config [options]

This section handles the usage of the config command. See Configuring axe-scan section for more details on the respective configuration items.

Check the valid axe-scan settings. When used with the --change-value option, it will modify the axe-scan setting to the local ./axe-scan.config.json file. Returns an error if local axe-scan.config.json does not exist and encourages the user to run axe-scan init first.

To assign multiple values to a key, connect the values with a comma:

axe-scan config -V resultTypes=incomplete,violations,passes

Options

Run the scan

axe-scan run [options]

Run the accessibility test. By default, axe-scan searches for urls.txt in the user's current directory and conducts accessibility tests against the URLs listed in the file. The contents of the text file should look something like the one below, where the target web page URLs are separated by line breaks. Text encoding should be in UTF-8.

https://www.website-to-test.com/
https://www.website-to-test.com/subpage/
https://www.website-to-test.com/another-subpage/

You may want to see also the Testing on websites with basic authentication section.

Axe-scan outputs the list of page-by-page, element-by-element test results, where the result type is violation or incomplete. See the axe-core documentation for their definitions. Which result type to output in the report can be changed by editing the configuration file axe-scan.config.json:

{
  "resultTypes": ["incomplete", "violations"] // Omitted from the default settings are "passes" and "inapplicable"
}

To create an output CSV file:

axe-scan run > axe-results.csv

Returns an error if urls.txt is not found. Optionally designate a custom path for the list of URLs using the --file option. Likewise, optionally designate an allowlist file path using the --allowlist flag. The matching alerts will be excluded from the output:

axe-scan run --file urls-list.txt --allowlist /path/to/allowlisted-alerts.csv > result-allowlisted.csv

For details on using the allowlist, see the Define allowlist section.

Options

Create a summarized report

axe-scan summary [options]

Create a summarized accessibility report of the web pages grouped by the WCAG success criteria. If the option --page is set, the report will be generated per page.

You may want to see also the Testing on websites with basic authentication section.

Options

Define allowlist

The results of axe-scan run are likely to come with items whose result type is incomplete, which refers to HTML elements that require further testing and manual checks. If, after further testing, you decide to regard the item as passes (= no accessibility issue found), you can add it to the allowlist.

The allowlist.csv is essentially a subset of results.csv created by:

axe-scan run > results.csv

Filter the rows that you would like to add to the allowlist and save it as another CSV file, e.g., allowlist.csv. Be sure to copy the first row, i.e., the header row, to the allowlist as well.

You can then run subsequent accessibility tests using the --allowlist <allowlistFilePath> option to filter out the items listed in the allowlist from the axe-scan outputs:

axe-scan run --allowlist allowlist.csv > results-allowlisted.csv
axe-scan summary -A allowlist.csv > summary.csv

Configuring axe-scan

axeCoreTags

The axe-core tags define the set of rules the accessibility test should be conducted on. For example, if you want to test the rules pertaining to WCAG 2.1 Level AA, you should include the wcag21aa tag. You can also designate a specific WCAG success criterion like wcag111, which indicates WCAG 2 success criterion 1.1.1. See the official documentation on axe-core tags for more detail and the possible values it can take.

Default value

["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"]

resultTypes

The results types of axe that should be output on axe-scan run command. The original axe results are grouped according to their outcome into the following arrays groups: passes (These results indicate what elements passed the rules), violations (These results indicate what elements failed the rules), inapplicable (These results indicate which rules did not run because no matching content was found on the page. For example, with no video, those rules won't run.), and incomplete (These results were aborted and require further testing. This can happen either because of technical restrictions to what the rule can test, or because a javascript error occurred.). See the official documentation on axe-core options for more detail.

Default value

["incomplete", "violations"]

filePath

File path to the list of URLs to test accessibility on. For example, if you want axe-scan to refer to the target-urls.txt in a subdirectory named data in your macOS, you would set this value to ./data/target-urls.txt.

Default value

./urls.txt

locale

The language of the axe output and messages of axe-scan. See dequelabs/axe-core for the comprehensive list of supported locales in axe. Except for en (English; the default value), the language code should match the JSON file names available in the locales folder of axe-core, e.g., de (Germany) from de.json and pt_BR (Portuguese (Brazilian)) from pt_BR.json. Note that messages of axe-scan are separately maintained in ./src/messages.ts of this repository

Default value

en

Testing on websites with basic authentication

When testing on websites with basic authentication, include the auth info in the list of URLs in the following format:

https://userid:password@example.com/

For example, if you are testing on the following pages:

https://mywebsite.com/
https://mywebsite.com/subpage1/
https://mywebsite.com/subpage2/

with basic authentication of ID: myAccountId and password: qwerty, your urls.txt should look like this:

https://myAccountId:qwerty@mywebsite.com/
https://myAccountId:qwerty@mywebsite.com/subpage1/
https://myAccountId:qwerty@mywebsite.com/subpage2/

Acknowledgments

Although coded from scratch, this tool itself was greatly inspired by works of louis-reed/axe-reports and caztcha/axe-test. Both have utilized the results of axe-core to create a CSV or spreadsheet of accessibility issues.