"AssertionError: 1 accessibility violation was detected: expected 1 to equal 0"?

[Pomax]

I'm using cypress-axe to do some accessibility testing, but the errors I'm getting are not clear enough for me to act on... Is there a way to get more detailed information here?

  6 passing (23s)
  22 pending
  2 failing

  1) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
     AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
      at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
      at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
      at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
      at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
      at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
      at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
      at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
      at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
      at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)

  2) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
     AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
      at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
      at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
      at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
      at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
      at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
      at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
      at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
      at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
      at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)

The test code is just a checka11y() call:

[...]

      context("Petition", () => {
        before(() => {
          cy.viewport('macbook-13');
          cy.visit(`/campaigns/single-page/`);
          cy.wait(500);
          cy.injectAxe().configureAxe({
            reporter: "v2"
          });
          cy.get(".sign-petition");
        });

        it("Petition content should be accessible.", () => {
          cy.checkA11y(".sign-petition .petition-content"); // passes
        });

        it("Petition form should be accessible.", () => {
          cy.checkA11y(".sign-petition form"); // generates those 2 errorrs
        });

[...]

And I updated cypress' /plugins/index.js to include the task logging:

module.exports = (on, config) => {
  on("task", {
    log(message) {
      console.log(message);
      return null;
    },
    table(message) {
      console.table(message);
      return null;
    }
  });
};

But I don't see anything in the console that lets me figure out what is actually failing and how to fix it =S

[austinezell]

I encountered this same problem recently. I found that the cy.checka11y can take a violations callback and that Cyrpress will output any thrown exception.

cy.checka11y(null, null, violations => { 
  // format violations
  // manually throw violations
})

[avanslaars]

Yup, you'll want to use the callback here. Defining those tasks registers them with Cypress, but using them requires that you call them with cy.task in the callback.

There is an example that was recently added to the readme here: https://github.com/avanslaars/cypress-axe#in-your-spec-file

[avanslaars]

Seems like this is resolved with @austinezell's comment. I'm going to close this issue.

[Pomax]

I'd strongly encourage you to reopen this, and only close it after adding a bit to the README that explains how to do this, because this is going to trip up a lot of folks who aren't in the habit of filing issues, and they won't find this information in a closed issue, because of SEO on the part of Google and friends.

Docs are the first port of call =)

[avanslaars]

There are instructions for this in the readme, right here: Using The Violation Callback

If you feel they could be improved, feel free to submit a PR

[Pomax]

fair enough