Section508Coordinators / Dev-Automation

This repository contains test automation scripts and related code arranged in examples for the integration of Section 508/accessibility in test automation activities within the Software Engineering Life Cycle. Examples span from the low to the high level of complexity of functionality and higher end test functionality and dependencies
https://section508coordinators.github.io/Dev-Automation/
MIT License
4 stars 5 forks source link

pa11y-ci test exit status 2 #1

Open danseo-reisys opened 2 years ago

danseo-reisys commented 2 years ago

Hello, trying out this sample script on our jenkins job.

The test always runs successfully however it always exits with status 2. ($?)

Is it because the scan found section 508 errors? Is there a way to catch the errors but ignore?

Thanks, Dan

https://github.com/Section508Coordinators/Dev-Automation/tree/master/examples/pa11y/pa11y-advanced-project1

2021-11-12 21:05:19 info: Starting script 2021-11-12 21:05:19 info: Getting configurion 2021-11-12 21:05:19 info: Getting template 2021-11-12 21:05:19 info: Loading sitemap into config 2021-11-12 21:05:19 info: Running pa11yCi... 2021-11-12 21:05:21 info: ...pa11yCi completed scanning all URLs 2021-11-12 21:05:21 info: Generating Html reports { "https://xxxxxxxxxxxxxx": [ { "code": "WCAG2AA.Principle3.Guideline3_2.3_2_2.H32.2", "type": "error", "typeCode": 1, "message": "This form does not contain a submit button, which creates issues for those who cannot submit the form using the keyboard. Submit buttons are INPUT elements with type attribute \"submit\" or \"image\", or BUTTON elements with type \"submit\" or omitted/invalid.", "context": "<form novalidate=\"\" id=\"searchForm-form\" class=\"search-form ng-untouched ng-pristine ng-valid\"><mat-form-field appearance=\"out...", "selector": "#searchForm-form", "runner": "htmlcs", "runnerExtras": {} }, { "code": "WCAG2AA.Principle4.Guideline4_1.4_1_1.F77", "type": "error", "typeCode": 1, "message": "Duplicate id attribute value \"mat-tab-group\" found on the web page.", "context": "<mat-tab-group _ngcontent-tfl-c216=\"\" id=\"mat-tab-group\" class=\"mat-tab-group remove-border-bottom ng-tns-c216-2 mat-primary\"><mat-tab-header class=\"mat-tab-...", "selector": "#mat-tab-group", "runner": "htmlcs", "runnerExtras": {} }, { "code": "WCAG2AA.Principle1.Guideline1_1.1_1_1.H37", "type": "error", "typeCode": 1, "message": "Img element missing an alt attribute. Use the alt attribute to specify a short text alternative.", "context": "<img _ngcontent-tfl-c212=\"\" fxflex=\"\" src=\"../assets/images/fema-logo.svg.png\" class=\"us-logo ng-tns-c212-0\" style=\"flex: 1 1 0%; box-sizing: border-box;\">", "selector": "#section-footer > mat-toolbar:nth-child(2) > div > div:nth-child(1) > img", "runner": "htmlcs", "runnerExtras": {} } ] } 2021-11-12 21:05:21 info: Getting results for html reports 2021-11-12 21:05:21 info: Generating page Html reports 2021-11-12 21:05:21 info: Generating Summary Html reports 2021-11-12 21:05:21 info: Completed script root@and-apis-fdx-portal-508-feature-feature-508-automation-24-b8j6z:/home/jenkins/agent/workspace/8-feature_feature-508-automation/508/pa11y/pa11y-advanced-project1/bin# echo $? 2

danseo-reisys commented 2 years ago

the following command always gives me exit status 2 and our jenkins stage ends prematurely.

node custom-pa11y.js --config config/01-custom-pa11y.config --template config/index.handlebars -h Reports