cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.09k stars 3.19k forks source link

spec is skipped if the `/runs/<runId>/instances` call fails #30628

Open mschile opened 6 days ago

mschile commented 6 days ago

Current behavior

If the /runs/<runId>/instances call fails and then succeeds, the current spec is skipped and execution moves onto the next spec but the app run doesn't fail or show any output from the first spec.

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.15.2                                                                        │
  │ Browser:        Chrome 131 (headless)                                                          │
  │ Node Version:   v20.18.0 (/Users/mschile/.nvm/versions/node/v20.18.0/bin/node)                 │
  │ Specs:          2 found (aliasing.cy.js, window.cy.js)                                         │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  │ Params:         Tag: false, Group: false, Parallel: false                                      │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

We encountered an unexpected error communicating with our servers.

RequestError: Error: ESOCKETTIMEDOUT

We will retry 3 more times in 30 seconds...

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  window.cy.js                                                                    (2 of 2)
  Estimated: 1 second

  Window
    ✓ cy.window() - get the global window object (395ms)
    ✓ cy.document() - get the document object (74ms)
    ✓ cy.title() - get the title (64ms)

  3 passing (1s)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        3                                                                                │
  │ Passing:      3                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 second                                                                         │
  │ Estimated:    1 second                                                                         │
  │ Spec Ran:     window.cy.js                                                                     │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

  (Uploading Cloud Artifacts)

  - Video - Nothing to upload 
  - Screenshot - Nothing to upload 
  - Test Replay - 520 kB

  Uploading Cloud Artifacts: . 

  (Uploaded Cloud Artifacts)

  - Test Replay - Done Uploading 520 kB in 670.89ms 1/1

====================================================================================================

  (Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  window.cy.js                             00:01        3        3        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:01        3        3        -        -        -  

───────────────────────────────────────────────────────────────────────────────────────────────────────

✨  Done in 102.55s.

Desired behavior

No response

Test code to reproduce

Cypress Version

13.15.2

Node version

v20.18.0

Operating System

macOS

Debug Logs

No response

Other

No response