pa11y / pa11y-ci

Pa11y CI is a CI-centric accessibility test runner, built using Pa11y
https://pa11y.org
GNU Lesser General Public License v3.0
515 stars 63 forks source link

pa11y headers configuration problem #181

Closed BioSs54 closed 1 year ago

BioSs54 commented 2 years ago

Hello,

pa11y gives me an error with no way to debug:

Analyzing 2 URLs:
 URL                                                                                              Errors    Warnings    Notices
 chrome-error://chromewebdata/                                                          2           0          0
 chrome-error://chromewebdata/                                                          2           0          0

Looks like this error is directly related to using the "headers" parameter.

I would like to know if any of you use this parameter and if so, how?

Here is my config file:

{
  "standard": "WCAG2AAA",
  "level": "error",
  "defaults": {
    "chromeLaunchConfig": {
      "args": [
        "--disable-dev-shm-usage",
        "--no-sandbox",
        "--disable-gpu"
      ],
      "ignoreHTTPSErrors": true
    },
    "headers": {
      "CT_REMOTE_USER": "username",
    },
    "includeWarnings": true,
    "reporters": [
      "pa11y-ci-reporter-cli-summary",
      ["json", { "fileName": "./pa11y/pa11y-ci-results.json" }],
      ["pa11y-ci-reporter-html", { "destination": "./pa11y" }]
    ],
    "timeout": 20000,
    "wait": 10000,
    "ignore": ["WCAG2AA.Principle3.Guideline3_2.3_2_2.H32.2"]
  },
  "urls": [
    "https://XXXXXXXXXXXXXX",
    "https://XXXXXXXXXXXXXX"
  ]
}

Thank you

aarongoldenthal commented 2 years ago

Hi @BioSs54,

I'm not sure about the headers question specifically, but your best starting point may be to try running in pa11y instead of pa11y-ci on one of the URLs. The pa11y-ci reporters only provide the error message when there is an execution error, not the full error with stack trace (as the author of two of those reporters, that's really driven by the most common errors having a cause that's a little more obvious - DNS errors, timeout ,etc). Plus, pa11y has a debug logging option that pa11y-ci does not, which may also help.

You'll need to create a pa11y compatible config file, which is really just the defaults object of your config above (without reporters, which is pa11y-ci specific), something like:

{
  "chromeLaunchConfig": {
    "args": ["--disable-dev-shm-usage", "--no-sandbox", "--disable-gpu"],
    "ignoreHTTPSErrors": true
  },
  "headers": { "CT_REMOTE_USER": "username" },
  "includeWarnings": true,
  "timeout": 20000,
  "wait": 10000,
  "ignore": ["WCAG2AA.Principle3.Guideline3_2.3_2_2.H32.2"]
}

and then run something like pa11y --config <your-config-file> --debug https://....

I can't imagine this is the source of the issue, but another observation from looking at your config. When pa11y-ci loads and processes configuration, it only passes on the urls and defaults properties. So, in the example above, the standard property should be within defaults. Unfortunately, the levels option is currently only used by pa11y, not pa11y-ci (and then only via the CLI). See https://github.com/pa11y/pa11y-ci/issues/94.

josebolos commented 1 year ago

Hi @BioSs54! I hope that @aarongoldenthal's tips helped you debug this.