Accessibility Assessment on Application Baseline for Pre-Phase II.

[binwang89]

As an application developer I would like a baseline report of the application's current accessibility compliance, to use for reference when determining level of effort for accessibility remediation.

Tasks

• [x] Using an accessibility testing tool, generate a11y issue reports for each page in the application (WCAG2AA Standards)
• [x] Provide a summary of findings, which will include additional notes on issues not covered by the testing tool
• [x] Provide the scan reports

[mmurthydol]

@jmmcnj @binwang89 Must not be started till after Brendan's AngularJS accessibility changes are made. My understanding was that this work was being done by DOL not by AIS. Wondering if I am missing something??

[ltsheu]

We have completed the accessibility assessment and written up a summary of findings. Please see the following two documents.

The Summary document provides an overview of the scan results, as well as notes on accessibility issues not covered by the scan. Section14cBaselineA11y-Summary.pdf

The Reports document provides details on the accessibility scans, performed with the aXe tool. Section14cBaselineA11y-aXeReports.pdf

[binwang89]

@ltsheu awesome !

[jmmcnj]

These are scanned as of master branch source - last year build in old staging see URL in pdf attached

[binwang89]

@ltsheu @simplyshang I have reviewed the detail report, that detail report is not show the detail information. I have also running the aXe myself against to DOL DEV environment. we should be able to generated the detail information for every issue show on the axe scan result, correct ?

[jmmcnj]

Reopening for further discussion, @binwang89 is asking for more details in the report, she is going to discuss with @simplyshang and @ltsheu to make sure she gets what she wants and understands the effort needed to get the detail she wants

[binwang89]

@ltsheu Hi, I put below screenshot as example that axe will display more detail information when you click on each issue item. for example, Element source/location will be benefit to part of the detail report.

[ltsheu]

@binwang89 @jmmcnj @simplyshang Hi! For the purposes of the baseline scan, we felt that capturing the amount and type of errors would be sufficient. aXe does not provide reports to an output document. Therefore capturing each error detail would require manually navigating through the in-browser details for each error and taking screenshots of the details, then compiling those screenshots into a document.

I had hoped to use Brendan's pa11y scan script to get past authentication and generate reports, however at the time last week that I was doing this work, the script did not output to a document. It generated a list (of errors only), within the command prompt. Therefore we decided to use an in-browser scanning tool (aXe) to get past authentication and capture the breadth of the issues. However, from our meeting with Brendan yesterday, I remember he said he has made improvements to that script, and he might also be adding the ability to output the report to a document of some kind. If this is the case, I would suggest we generate the reports using the new pa11y script instead of aXe.

Moving forward, aXe is useful for the developers to review details of specific accessibility errors within the aXe browser interface, on an error-by-error basis.

[mgwalker]

Brendan's updates to the pa11y script to do that are in #268, and I just merged them into the sprint/1 branch.

[ltsheu]

Great! @simplyshang and I will take a look at the new pa11y script on Monday, and see what kind of report we can get with it. We'll post an update here.

[binwang89]

@ltsheu if you can use the pally script on Monday and generate the report, that will be great.

[ltsheu]

@binwang89 If the pa11y script works well and we can generate the reports, would you prefer that I run the scan on the pre-Phase II baseline application (https://dol-whd-section14c-stg.azurewebsites.net)? Or should I run the scan on the most recent version of the application which would include Brendan's a11y updates (sprint/1 branch)? Thanks.

[binwang89]

@ltsheu Good Point. i think it is make sense to run on the most recent version of the application which would include Brendan's a11y updates. Let's also discuss it on Monday's 3 pm UX meeting. Thanks

[ltsheu]

@binwang89 Here is an example of a report from the pa11yScan.js script. The script has an option to output to JSON, but I felt for this purpose an HTML output would be better. So I added the HTML option to pa11yScan.js. @mgwalker I will wait to do a pull request as there may be more discussion about these pa11y reports.

In the meantime, here is a PDF report for the Application Info screen (generated from the HTML report). Please let me know if the report is acceptable - we can also discuss in 3pm UX meeting. Thanks!

pa11y-results-AppInfo-10-02-17.pdf

[binwang89]

@ltsheu does the pallyScan.js script can provide the issue location to the output report when scan the site?

[ltsheu]

@binwang89 Currently it gives clues to the location of each issue by showing the HTML tag and sometimes a hint of where it is in the DOM. I could research more to see if there are other options in pa11y for location information. For example are you looking for it to output a specific line number? I'm not sure it's possible but I could do some research.

In-browser tools such as aXe and HTML Code Sniffer do a better job of pointing to the location of each error, which is certainly useful for developers when they are remediating the issues. However, these tools don't output the reports to a file.

[ltsheu]

I have run a pa11y scan on each page. The scan reports are available in sprint/1 branch: https://github.com/18F/dol-whd-14c/tree/sprint/1/docs/a11y. I will move this user story to the Review stage.