pa11y / pa11y-ci

Pa11y CI is a CI-centric accessibility test runner, built using Pa11y
https://pa11y.org
GNU Lesser General Public License v3.0
519 stars 63 forks source link

Page crash errors when running against a Jekyll site #128

Closed dmundra closed 2 years ago

dmundra commented 3 years ago

Hi there,

I have noticed that when running pa11y-ci locally or in a GitHub action I will get these errors

> http://localhost:4000/people/jennifer-aube - 0 errors
(node:1918) UnhandledPromiseRejectionWarning: Error: Page crashed!
    at Page._onTargetCrashed (/home/runner/work/accessibility/accessibility/node_modules/puppeteer/lib/Page.js:215:24)
    at CDPSession.<anonymous> (/home/runner/work/accessibility/accessibility/node_modules/puppeteer/lib/Page.js:123:56)
    at CDPSession.emit (events.js:315:20)
    at CDPSession._onMessage (/home/runner/work/accessibility/accessibility/node_modules/puppeteer/lib/Connection.js:200:12)
    at Connection._onMessage (/home/runner/work/accessibility/accessibility/node_modules/puppeteer/lib/Connection.js:112:17)
    at WebSocket.<anonymous> (/home/runner/work/accessibility/accessibility/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)
    at WebSocket.onMessage (/home/runner/work/accessibility/accessibility/node_modules/ws/lib/event-target.js:120:16)
    at WebSocket.emit (events.js:315:20)
    at Receiver.receiverOnMessage (/home/runner/work/accessibility/accessibility/node_modules/ws/lib/websocket.js:789:20)
    at Receiver.emit (events.js:315:20)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1918) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:1918) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Do you have a recommendation of something I could try to avoid these errors. Anyone else has gotten these?

Thank you

ssh6 commented 3 years ago

@dmundra I'm getting the same error, but intermittently. Running 2.4.0 in a docker container.

(node:1) UnhandledPromiseRejectionWarning: Error: Page crashed!
    at Page._onTargetCrashed (/usr/local/lib/node_modules/pa11y-ci/node_modules/puppeteer/lib/Page.js:215:24)
    at CDPSession.Page.client.on.event (/usr/local/lib/node_modules/pa11y-ci/node_modules/puppeteer/lib/Page.js:123:56)
    at CDPSession.emit (events.js:182:13)
    at CDPSession._onMessage (/usr/local/lib/node_modules/pa11y-ci/node_modules/puppeteer/lib/Connection.js:200:12)
    at Connection._onMessage (/usr/local/lib/node_modules/pa11y-ci/node_modules/puppeteer/lib/Connection.js:112:17)
    at WebSocketTransport._ws.addEventListener.event (/usr/local/lib/node_modules/pa11y-ci/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)
    at WebSocket.onMessage (/usr/local/lib/node_modules/pa11y-ci/node_modules/ws/lib/event-target.js:120:16)
    at WebSocket.emit (events.js:182:13)
    at Receiver.receiverOnMessage (/usr/local/lib/node_modules/pa11y-ci/node_modules/ws/lib/websocket.js:789:20)
    at Receiver.emit (events.js:182:13)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

• Error: Pa11y timed out (60000ms) (note: I don't think this is correct)

"defaults": {
  "concurrency": "8",
  "chromeLaunchConfig": {
     "ignoreHTTPSErrors": true,
     "args": ["--no-sandbox"]
   },
    "timeout": 60000
},
ssh6 commented 3 years ago

probably related to https://jonasjancarik.medium.com/handling-those-unhandled-promise-rejections-when-using-javascript-async-await-and-ifee-5bac52a0b29f and #111

dmundra commented 3 years ago

Good find @ssh6. I will take a look.

ssh6 commented 3 years ago

switching to older pa11y-ci (2.1.1) may be helping. Hard to troubleshoot intermittent problems though.

dmundra commented 3 years ago

Hey, @ssh6 is the older version more stable?

ssh6 commented 3 years ago

@dmundra doesn't appear to be. And the older version interprets results differently as well. So I would probably stick with the new version and hope for a magic update that fixes everything. (To "fix" my problem, I dropped the page in question from testing.)

josebolos commented 3 years ago

Hi @dmundra and @ssh6

Could you please provide a URL that triggers this issue, even if it is occasionally? There's no way that we can replicate this issue otherwise.

ssh6 commented 3 years ago

@josebolos https://chimes.cornell.edu/Chimesmasters was the item I removed from my tests.

dmundra commented 3 years ago

@josebolos here is an example of a recent pull request with the error: https://github.com/CivicActions/accessibility/pull/419

We run pa11y-ci on every commit in a pull request.

josebolos commented 3 years ago

Thanks both! I'll try to see if I can replicate it.

hursey013 commented 3 years ago

In case you need another example: https://github.com/usdoj-crt/beta-ada/runs/2603206599

josebolos commented 3 years ago

Thanks all, I've been able to replicate the problem so I've opened a new issue in the pa11y repo: https://github.com/pa11y/pa11y/issues/570

Feel free to subscribe to that issue for updates.

I will keep this one open until a fix is released in pa11y-ci anyway.

josebolos commented 2 years ago

Version 3.0.0 of pa11y-ci is out which include several improvements (including a puppeteer upgrade) that should have gotten rid of this error. Please create a new issue if you stumble upon a URL that fails to run with v3.0.0.