search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.71k stars 3.16k forks source link

Flaky test: Screenshot Timeout system test in Firefox #29116

Open cacieprins opened 6 months ago

cacieprins commented 6 months ago

Link to Cypress Cloud or CircleCI failure

https://app.circleci.com/pipelines/github/cypress-io/cypress/60332/workflows/5722539d-d040-4f5b-9585-fbcba73ef471/jobs/2512142

Link to failing test in GitHub

https://github.com/cypress-io/cypress/blob/develop/system-tests/test/issue_5016_spec.js

Analysis

This system test was added to detect regressions due to a fix implemented for #5016. This test exhibits flake in Firefox, even when run against commits prior to the 5016 fix. The test that should be succeeding but fails in this system test clicks an anchor with target="_blank" and takes a screenshot of that anchor: https://github.com/cypress-io/cypress/blob/develop/system-tests/projects/config-screenshot-on-failure-enabled/cypress/e2e/issue-5016.cy.js

An unexpected erorr is thrown in the v2 extension:

  Error: An unexpected error occurred
      (No stack trace)
  From previous event:
      at takeScreenshot (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:20850:6)
      at ./app/v2/background.js/connect/invoke (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:20623:33)
  From previous event:
      at invoke (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:20622:20)
      at ./app/v2/background.js/connect (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:20651:16)
      at ../../node_modules/component-emitter/index.js/Emitter.prototype.emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:5736:20)
      at emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24813:15)
      at emitEvent (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24559:20)
      at onevent (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24546:18)
      at onpacket (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24510:22)
      at ../../node_modules/component-emitter/index.js/Emitter.prototype.emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:5736:20)
      at emitReserved (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24826:15)
      at ondecoded (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24096:14)
      at ../../node_modules/component-emitter/index.js/Emitter.prototype.emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:5736:20)
      at add (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:23404:23)
      at ondata (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:24088:22)
      at ../../node_modules/component-emitter/index.js/Emitter.prototype.emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:5736:20)
      at onPacket (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:21337:16)
      at ../socket/node_modules/engine.io-client/lib/socket.js/setTransport (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:21146:14)
      at ../../node_modules/component-emitter/index.js/Emitter.prototype.emit (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:5736:20)
      at onPacket (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:21738:10)
      at onData (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:21731:10)
      at ../socket/node_modules/engine.io-client/lib/transports/websocket.js/addEventListeners/this.ws.onmessage (moz-extension://701245c8-3fef-4fbb-aaae-a44f0c51660e/background.js:22722:36)

Cypress Version

13.6.7

Other

For now, we are disabling Firefox on this system test. This is likely indicative of .screenshot() intermittently failing with a timeout error in Firefox when the main Cypress tab is not focused.

cypress-app-bot commented 1 week ago

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.