A CLI tool to test web accessibility on multiple web pages based on a list of URLs in a text file. Run on Node.js and uses axe-core and Puppeteer as testing and browsing engines.
Making a website accessible is not just about social equity and legal compliance; accessibility is fundamental to developing and maintaining a user-oriented website. Improved user experience is likely to be favored by search engines like Google, which should be a heads-up for the management-level members of the organization who are less interested in this topic.
Web accessibility is often brought up in the context of the responsibilities of designers and developers, and indeed, the Web Content Accessibility Guidelines (WCAG) state itself that it is intended for:
- Web content developers (page authors, site designers, etc.)
- Web authoring tool developers
- Web accessibility evaluation tool developers
- Others who want or need a standard for web accessibility, including for mobile accessibility
(Who WCAG is for from WCAG 2.0 Overview)
Looking a little deeper into the actual contents of the WCAG, however, you will see that EVERY member of those related to developing and maintaining a website, not just the developers and engineers, need to be aware of the requirements to achieve a certain level of accessibility and keep that level. Axe-scan aims to provide a common ground of discussion to improve the website's accessibility between tech professionals and other members who are less familiar with WCAG through the following features:
incomplete
, which refers to HTML elements that axe-core could not mechanically determine as passes
or violations
of the rules. These incomplete
results may include elements that do not need further action. To avoid such elements from repetitively appearing in the results, the axe-scan provides an option to define the list of the elements as an allowlist.This section lays out the basic outlines of testing web accessibility using axe-scan. See the later sections for details.
See the Installation section.
Setup axe-scan in your working directory:
axe-scan init
If you want to change the language of the axe-scan output:
axe-scan config --change-value locale=ja
The above example sets the locale to ja
or Japanese. See the locale
in Configuring axe-scan section for more details on the values that it can take.
urls.txt
In your current working directory, create a list of the URLs on which to conduct the accessibility test. The URLs should be separated by line breaks. Name the file urls.txt
. See Run the scan section for more detail.
Conduct the initial test and save the results to results.csv
:
axe-scan run > results.csv
A list of around 1,000 pages will take approx. 1 hour to complete.
Open the results.csv
and check the contents. You will likely see that under the Result Types
column, there are violations
and incomplete
items.
Violations
are elements that clearly violate one or more of the WCAG Success Criteria. Address these issues. Note that there may be multiple rows for a single issue if it has more than one way to resolve it. Once handled properly using one of the methods, the set of items will all disappear.
Incomplete
are elements that axe-core could not clearly determine whether it is a violation or not. Check the item manually and determine if it is a violation or not. If no problems were found, flag the row to include it in the allowlist for future use.
Create an allowlist.csv
by filtering the results.csv
for items that you marked so. Include the first row, i.e., the header row, when you save allowlist.csv
as a new file.
Execute the second run on the accessibility test, this time taking into account the contents of the allowlist.csv
:
axe-scan run --allowlist allowlist.csv > results-allowlisted.csv
Check the created CSV file results-allowlisted.csv
and see that items in the allowlist.csv
are not present. In fact, if you have resolved the violations in the previous step, this CSV file should be empty.
To create a summary report based on the WCAG Success Criteria, run:
axe-scan summary --allowlist allowlist.csv > summary-allowlisted.csv
After the previous steps, all items should be PASS or INAPPLICABLE.
results-allowlisted.csv
allowlist.csv
summary-allowlisted.csv
You can use the allowlist.csv
for periodical web accessibility tests using axe-scan to easily detect new violations
and incomplete
items.
Axe-scan is not a one-size-fits-all accessibility test. There are some WCAG Success Criteria (SC) that axe-scan cannot check, so to create a full checklist of all the WCAG SC, you will have to use axe-scan together with other means like manual reviews or tests using other specialized tools.
To install axe-scan, you must have Node.js installed as a prerequisite. To check, enter on your Terminal, Powershell, Command Prompt, or your preferred client:
node -v
If this returns an error, install the latest LTS version of Node.js before you proceed below. This document will not go into the details, but there are ways to make version management of Node.js easier for macOS and Linux users:
On your Terminal, enter:
npm install -g axe-scan
This installs axe-scan globally, i.e., all executions run on your account will use the same version of axe-scan.
If you want to separate the versions depending on your working directory, install axe-scan locally without the -g
option:
npm install axe-scan
If you want to install a specific version of axe-scan, add the version number to the package name:
npm install axe-scan@1.0.0
To update your installed axe-scan:
npm update -g axe-scan
Remove the -g
option like in the Install axe-scan section, depending on your needs.
axe-scan init [options]
Create a axe-scan.config.json
in the working directory with the default values. If there is an existing config file in the directory, it will ask you whether to update the file or to abort. If you have placed a config file at your home directory ~/axe-scan.config.json
, axe-scan init
will copy that setting instead of the default values.
Use the --global
option to create the config file in your home directory rather than your current working directory:
axe-scan init --global
-g
, --global
: Create the config file in the home directory.{
"axeCoreTags": ["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"],
"resultTypes": ["incomplete", "violations"],
"filePath": "./urls.txt",
"locale": "en"
}
See Configuring axe-scan section for more detail. User settings will be determined in the following order: (1) ./axe-scan.config.json
file in the working directory, (2) ~/axe-scan.config.json
file in the user's home directory, and (3) default settings.
axe-scan config [options]
This section handles the usage of the
config
command. See Configuring axe-scan section for more details on the respective configuration items.
Check the valid axe-scan settings. When used with the --change-value
option, it will modify the axe-scan setting to the local ./axe-scan.config.json
file. Returns an error if local axe-scan.config.json
does not exist and encourages the user to run axe-scan init
first.
To assign multiple values to a key, connect the values with a comma:
axe-scan config -V resultTypes=incomplete,violations,passes
-V
, --change-value <key>=<value>
: Modify the value of the designated key item in the configuration file.-g
, --global
: Edit the config file in the home directory instead of the current directory. Use with the --change-value
option.axe-scan run [options]
Run the accessibility test. By default, axe-scan searches for urls.txt
in the user's current directory and conducts accessibility tests against the URLs listed in the file. The contents of the text file should look something like the one below, where the target web page URLs are separated by line breaks. Text encoding should be in UTF-8.
https://www.website-to-test.com/
https://www.website-to-test.com/subpage/
https://www.website-to-test.com/another-subpage/
You may want to see also the Testing on websites with basic authentication section.
Axe-scan outputs the list of page-by-page, element-by-element test results, where the result type is violation
or incomplete
. See the axe-core documentation for their definitions. Which result type to output in the report can be changed by editing the configuration file axe-scan.config.json
:
{
"resultTypes": ["incomplete", "violations"] // Omitted from the default settings are "passes" and "inapplicable"
}
To create an output CSV file:
axe-scan run > axe-results.csv
Returns an error if urls.txt
is not found. Optionally designate a custom path for the list of URLs using the --file
option. Likewise, optionally designate an allowlist file path using the --allowlist
flag. The matching alerts will be excluded from the output:
axe-scan run --file urls-list.txt --allowlist /path/to/allowlisted-alerts.csv > result-allowlisted.csv
For details on using the allowlist, see the Define allowlist section.
-F
, --file <urlsFilePath>
: Designate the file path for the list of URLs on which to conduct the accessibility test.-A
, --allowlist <allowlistFilePath>
: Designate the file path for the allowlist of accessibility alerts.axe-scan summary [options]
Create a summarized accessibility report of the web pages grouped by the WCAG success criteria. If the option --page
is set, the report will be generated per page.
You may want to see also the Testing on websites with basic authentication section.
-F
, --file <urlsFilePath>
: Designate the file path for the list of URLs on which to conduct the accessibility test.-P
, --page
: Create the summary report on per page basis.-A
, --allowlist <allowlistFilePath>
: Designate the file path for the allowlist of accessibility alerts.The results of axe-scan run
are likely to come with items whose result type is incomplete
, which refers to HTML elements that require further testing and manual checks. If, after further testing, you decide to regard the item as passes
(= no accessibility issue found), you can add it to the allowlist.
The allowlist.csv
is essentially a subset of results.csv
created by:
axe-scan run > results.csv
Filter the rows that you would like to add to the allowlist and save it as another CSV file, e.g., allowlist.csv
. Be sure to copy the first row, i.e., the header row, to the allowlist as well.
You can then run subsequent accessibility tests using the --allowlist <allowlistFilePath>
option to filter out the items listed in the allowlist from the axe-scan outputs:
axe-scan run --allowlist allowlist.csv > results-allowlisted.csv
axe-scan summary -A allowlist.csv > summary.csv
axeCoreTags
The axe-core tags define the set of rules the accessibility test should be conducted on. For example, if you want to test the rules pertaining to WCAG 2.1 Level AA, you should include the wcag21aa
tag. You can also designate a specific WCAG success criterion like wcag111
, which indicates WCAG 2 success criterion 1.1.1. See the official documentation on axe-core tags for more detail and the possible values it can take.
["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"]
resultTypes
The results types of axe that should be output on axe-scan run
command. The original axe results are grouped according to their outcome into the following arrays groups: passes
(These results indicate what elements passed the rules), violations
(These results indicate what elements failed the rules), inapplicable
(These results indicate which rules did not run because no matching content was found on the page. For example, with no video, those rules won't run.), and incomplete
(These results were aborted and require further testing. This can happen either because of technical restrictions to what the rule can test, or because a javascript error occurred.). See the official documentation on axe-core options for more detail.
["incomplete", "violations"]
filePath
File path to the list of URLs to test accessibility on. For example, if you want axe-scan to refer to the target-urls.txt
in a subdirectory named data
in your macOS, you would set this value to ./data/target-urls.txt
.
./urls.txt
locale
The language of the axe output and messages of axe-scan. See dequelabs/axe-core for the comprehensive list of supported locales in axe. Except for en
(English; the default value), the language code should match the JSON file names available in the locales folder of axe-core, e.g., de
(Germany) from de.json
and pt_BR
(Portuguese (Brazilian)) from pt_BR.json
. Note that messages of axe-scan are separately maintained in ./src/messages.ts
of this repository
en
When testing on websites with basic authentication, include the auth info in the list of URLs in the following format:
https://userid:password@example.com/
For example, if you are testing on the following pages:
https://mywebsite.com/
https://mywebsite.com/subpage1/
https://mywebsite.com/subpage2/
with basic authentication of ID: myAccountId
and password: qwerty
, your urls.txt
should look like this:
https://myAccountId:qwerty@mywebsite.com/
https://myAccountId:qwerty@mywebsite.com/subpage1/
https://myAccountId:qwerty@mywebsite.com/subpage2/
Although coded from scratch, this tool itself was greatly inspired by works of louis-reed/axe-reports and caztcha/axe-test. Both have utilized the results of axe-core to create a CSV or spreadsheet of accessibility issues.