mfrachet / cypress-audit

⚡ Run Lighthouse and Pa11y audits directly in your E2E test suites
https://mfrachet.github.io/cypress-audit/
MIT License
351 stars 44 forks source link

cypress-audit not working on Jenkins, always return 0 score #237

Open Esteban-Rocha opened 1 month ago

Esteban-Rocha commented 1 month ago

What does not work? When running on a Jenkins pipeline, the result score is always 0, locally it works perfectly fine:

Failing pipeline on Jenkins:

cy:command ✔ uncaught exception    TypeError: Cannot read properties of null (reading 'contentWindow')
cy:command ✔ url   
cy:log ⓘ  -------- cy.lighthouse --------
cy:command ✔ task  lighthouse, Object{4}
cy:log ⓘ -----------------------------
cy:command ✔  wrap performance record is 0 and is under the 85 threshold
cy:command ✘ then function(){}

"Running" pipeline locally:

cy:command ✔ uncaught exception    TypeError: Cannot read properties of null (reading 'contentWindow')
cy:command ✔ url   
cy:log ⓘ  -------- cy.lighthouse --------
cy:command ✔ task  lighthouse, Object{4}
cy:log ⓘ -----------------------------
cy:command ✔  wrap performance record is 81 and is under the 85 threshold
cy:command ✘ then function(){}

As you can see, the log is identical except for the actual score.

Expected behavior The scores are the same when running the test suite, they should not drop to 0 when running on the pipeline.

Environment (please complete the following information):