GitHub Action: runs Axe accessibility tests on changed files

[jcarstairs-scottlogic]

This PR implements a GitHub Action to check for low-hanging accessibility problems in new blog posts. Here's an example workflow showing the tests running and failing because of accessibility issues.

To be more precise, the Action uses the pa11y-ci command-line interface to run the Axe accessibility checks against a list of new or changed pages.

I've written a bash script to calculate the URLs of the pages which have changed, which won't catch all changes, but will catch the vast majority in a way which is both simple and compute-cheap, and in particular will catch new blog posts.

The Action is triggered by Pull Requests and pushes to gh-pages.

Running the tests only against the changed pages is quicker and uses less compute. To figure out which pages have changed in a way which is fairly simple and doesn't require too much compute, I've written a script, generate_pa11y_ci_urls_from_git_diff, which calculates the relative URLs of the pages which have changed given a Git diff against a provided base. This approach misses out on pages which have changed as a result of changes to Jekyll layouts and includes, but since the vast majority of changes are new blog posts, this approach should be good enough.

Since colour contrast is a known issue, I've configured aXe to ignore colour contrast issues.

[ColinEberhardt]

Thanks @jcarstairs-scottlogic looks like a good idea - if I merge at the moment, will we be error free?

[jcarstairs-scottlogic]

Thanks @jcarstairs-scottlogic looks like a good idea - if I merge at the moment, will we be error free?

I guess you're asking whether the gh-pages branch will pass the tests? If so, the tl;rd is: N/A, because the pipeline won't run on gh-pages.

Actually, I just fixed this: before the last commit, the pipeline would have been triggered by pushes to gh-pages! That was a mistake. Now, it only gets triggered by PRs.

It makes sense to trigger the pipeline on PRs, not the gh-pages branch, because the idea is to spot regressions in new or edited content. This pipeline isn't meant for finding existing defects.

I hope that answers your question?

[jcarstairs-scottlogic]

I should also highlight again the demo of the pipeline running on an example change I linked in the PR description. You can see how the pipeline only runs tests on 8 URLs, namely, the ones where I added in some fake blog posts. It fails because the new posts have accessibility defects. If you want an example of the pipeline passing, have a look at the running on this branch head.