[CSR-791] cypress-cloud is not compatible with cypress 13.x

[anlambert]

Before opening, please confirm:

• [X] I have searched for duplicate or closed issues and discussions.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• [X] I acknowledge that I will attach a full debug log, otherwise the issue will be closed with no response.

Environment information

``` # Put output below this line System: OS: Linux 6.1 Debian GNU/Linux 12 (bookworm) 12 (bookworm) CPU: (8) x64 Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz Memory: 18.94 GB / 31.16 GB Container: Yes Shell: 5.2.15 - /bin/bash Binaries: Node: 18.16.0 - /usr/bin/node Yarn: 1.22.19 - /usr/bin/yarn npm: 9.5.1 - /usr/bin/npm Browsers: Chrome: 114.0.5735.90 Chromium: 116.0.5845.140 npmPackages: eslint-config-custom: latest => 0.0.0 prettier: latest => 3.0.2 turbo: ^1.10.13 => 1.10.13 npmGlobalPackages: azurite: 3.23.0 corepack: 0.17.0 gulp-cli: 2.3.0 npm: 9.5.1 samfirm: 0.2.0 ```

Describe the bug

Due to this cypress commit that introduces some breaking API changes in cypress 13.x, I noticed two issues when using cypress-cloud with it:

1. Executed cypress tests are always reported as failing whatever their real statuses are.

See sample cypress-cloud output put below:

``` $ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36 yarn run v1.22.19 $ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36 WARNING Failed to load config file, falling back to the default config. Attempted locations: [ 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js', 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs', 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs' ] Copyright (C) 2023 Currents Software Inc https://currents.dev This is free software, and you are welcome to redistribute it under certain conditions. This program comes with no warranty. Parts of this program are MIT licensed. Refer to the license for details https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md Discovered 1 spec files Tags: false; Group: false; Parallel: true; Batch Size: 3 Connecting to cloud orchestration service... 🎥 Run URL: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7 ==================================================================================================== Running: cypress/e2e/home.cy.js (1/1) ==================================================================================================== (Run Starting) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 13.1.0 │ │ Browser: Electron 106 (headless) │ │ Node Version: v18.16.0 (/usr/bin/node) │ │ Specs: 1 found (home.cy.js) │ │ Searched: cypress/e2e/home.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ──────────────────────────────────────────────────────────────────────────────────────────────────── Running: home.cy.js (1 of 1) Home Page Tests ✓ should have focus on search form after page load ✓ should display positive stats for each category ✓ should display null counters and hide history graphs when storage is empty ✓ should hide counters when data is missing ✓ should redirect to search page when submitting search form 5 passing (6s) (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 5 │ │ Passing: 5 │ │ Failing: 0 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: false │ │ Duration: 6 seconds │ │ Spec Ran: home.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ==================================================================================================== (Run Finished) Spec Tests Passing Failing Pending Skipped ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ✔ home.cy.js 00:06 5 5 - - - │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ✔ All specs passed! 00:06 5 5 - - - Reporting results and artifacts in background... ==================================================================================================== Cloud Run Finished ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Spec Tests Passing Failing Pending Skipped │ ├────────────────────────────────────────────────────────────────────────────────────────────────┤ │ ✖ cypress/e2e/home.cy.js 0ms 1 - 1 - - │ ├────────────────────────────────────────────────────────────────────────────────────────────────┤ │ ✖ 1 of 1 failed 0ms 1 - 1 - - │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ 🏁 Recorded Run: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7 error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. ```

After reading cypress-cloud code, it seems that issue can be fixed using the following diff:

diff --git a/packages/cypress-cloud/lib/results/results.ts b/packages/cypress-cloud/lib/results/results.ts
index f9380de..635f0b4 100644
--- a/packages/cypress-cloud/lib/results/results.ts
+++ b/packages/cypress-cloud/lib/results/results.ts
@@ -12,11 +12,6 @@ import { ConfigState } from "../state";

 const debug = Debug("currents:results");

-export const isSuccessResult = (
-  result: CypressResult
-): result is CypressCommandLine.CypressRunResult => {
-  return result.status === "finished";
-};

 export const getScreenshotsSummary = (
   tests: CypressCommandLine.TestResult[] = []
@@ -282,9 +277,6 @@ export function getCypressRunResultForSpec(
   spec: string,
   cypressResult: CypressResult
 ): CypressCommandLine.CypressRunResult | undefined {
-  if (!isSuccessResult(cypressResult)) {
-    return;
-  }

   const run = cypressResult.runs.find((r) => r.spec.relative === spec);
   if (!run) {

2. Screenshots fail to be processed.

Once the fix above applied, there is an error when cypress-cloud attempts to process test screenshots, see output below:

``` $ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38 yarn run v1.22.19 $ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38 WARNING Failed to load config file, falling back to the default config. Attempted locations: [ 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js', 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs', 'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs' ] Copyright (C) 2023 Currents Software Inc https://currents.dev This is free software, and you are welcome to redistribute it under certain conditions. This program comes with no warranty. Parts of this program are MIT licensed. Refer to the license for details https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md Discovered 1 spec files Tags: false; Group: false; Parallel: true; Batch Size: 3 Connecting to cloud orchestration service... 🎥 Run URL: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae ==================================================================================================== Running: cypress/e2e/home.cy.js (1/1) ==================================================================================================== (Run Starting) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 13.1.0 │ │ Browser: Electron 106 (headless) │ │ Node Version: v18.16.0 (/usr/bin/node) │ │ Specs: 1 found (home.cy.js) │ │ Searched: cypress/e2e/home.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ──────────────────────────────────────────────────────────────────────────────────────────────────── Running: home.cy.js (1 of 1) Home Page Tests ✓ should have focus on search form after page load ✓ should display positive stats for each category ✓ should display null counters and hide history graphs when storage is empty ✓ should hide counters when data is missing ✓ should redirect to search page when submitting search form 5 passing (6s) (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 5 │ │ Passing: 5 │ │ Failing: 0 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: false │ │ Duration: 6 seconds │ │ Spec Ran: home.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ==================================================================================================== (Run Finished) Spec Tests Passing Failing Pending Skipped ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ✔ home.cy.js 00:06 5 5 - - - │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ✔ All specs passed! 00:06 5 5 - - - Reporting results and artifacts in background... ERROR TypeError: Cannot read properties of undefined (reading 'flatMap') at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:26:21 at Array.flatMap () at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:25:19 at Array.flatMap () at getScreenshotsSummary (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:24:16) at getInstanceResultPayload (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:62:18) at getReportResultsTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/uploadResults.ts:22:13) at createReportTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/reportTask.ts:26:3) at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:55:7 at Array.forEach () at runTillDone (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:54:7) ==================================================================================================== Cloud Run Finished ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Spec Tests Passing Failing Pending Skipped │ ├────────────────────────────────────────────────────────────────────────────────────────────────┤ │ ✔ undefined 6.2s 5 5 - - - │ ├────────────────────────────────────────────────────────────────────────────────────────────────┤ │ ✔ All specs passed! 6.2s 5 5 - - - │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ 🏁 Recorded Run: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae Done in 13.82s. ```

The fix for this one is not straightforward to me.

Expected behavior

cypress 13.x should be supported.

Command and Setup

Just run any cypress-cloud session with cypress 13.x and the reported errors will show up.

Full log and debug output

cypress_cloud_debug_logs.txt

[agoldis]

The fix is in progress

[Roemer]

Is this also related to the issue that viewing a run in Sorry Cypress is not possible and shows the following error:

[agoldis]

Please check out the upgrade guide: https://currents.dev/readme/integration-with-cypress/cypress-cloud/migration-to-cypress-13

[anlambert]

I just tested the upgrade to cypress-cloud beta and tests are no longer reported as failing, see https://jenkins.softwareheritage.org/blue/organizations/jenkins/DWAPPS%2Fgitlab-builds/detail/gitlab-builds/302/pipeline/89/.

There is still the error about screenshots that appears after each test suite though but if you do not use that cypress feature it is not critical for running the tests.

[richardmward]

I too am having issues with the latest build 1.10.0-beta.2, but not the same as above.

file:///C:/dev/src/my-project/node_modules/cypress-cloud/index.mjs:2273
    ..._11.cloneDeep(specResults),
           ^

TypeError: _11.cloneDeep is not a function

Hacking in a bit of debugging, it seems that _11 contains default, which contains cloneDeep - which seems to me to be imported as a commonjs module rather than esm (which is how all my project is configured).

EDIT:

In v1.9.4, the imports in index.mjs are in the format import _11 from "lodash";, whereas in same file in v1.10.0-beta.2, the imports are now import * as _11 from "lodash";

[agoldis]

Thanks @anlambert and @richardmward, working on the fixes for those issues

[estefafdez]

For us, after updating everything to the latest version we got the following error, and the specs are not executed:

11:47:24  Copyright (C) 2023 Currents Software Inc https://currents.dev
11:47:24  This is free software, and you are welcome to redistribute it under certain
11:47:24  conditions. This program comes with no warranty. Parts of this program are MIT
11:47:24  licensed. Refer to the license for details
11:47:24  https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
11:47:24  
11:47:26  
11:47:26  
11:47:26    (Attempt 1 of 2) An uncaught error was detected outside of a test
11:47:26    1) An uncaught error was detected outside of a test
11:47:26  
11:47:26    0 passing (910ms)
11:47:26    1 failing
11:47:26  
11:47:26    1) An uncaught error was detected outside of a test:
11:47:26       Error: The following error originated from your test code, not from Cypress.
11:47:26  
11:47:26    > Dynamic require of "@neuralegion/cypress-har-generator/commands" is not supported
11:47:26  
11:47:26  When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
11:47:26  
11:47:26  Cypress could not associate this error to any specific test.
11:47:26  

Versions:

• "cypress": "13.1.0",
• "cypress-cloud": "1.9.4",
• "cypress-debugger": "^1.0.9",

[estefafdez]

The tests are working with

 "cypress": "13.1.0",
    "cypress-cloud": "1.10.0-beta.2",
    "cypress-debugger": "1.0.7",`

and you can see the logs on the command line that the tests passed ok:

12:45:25    (Run Starting)
12:45:25  
12:45:25    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:25    │ Cypress:        13.1.0                                                                         │
12:45:25    │ Browser:        Electron 106 (headless)                                                        │
12:45:25    │ Node Version:   v16.16.0 (/usr/bin/node)                                                       │
12:45:25    │ Specs:          1 found (admin_auth.spec.ts)                                                   │
12:45:25    │ Searched:       cypress/test/genially/admin/admin_auth.spec.ts                                 │
12:45:25    │ Experiments:    experimentalRunAllSpecs=true,experimentalMemoryManagement=true,experimentalWe… │
12:45:25    └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:25  
12:45:25  
12:45:25  ────────────────────────────────────────────────────────────────────────────────────────────────────
12:45:25                                                                                                      
12:45:25    Running:  admin_auth.spec.ts                                                              (1 of 1)
12:45:32  
12:45:32  
12:45:32    Admin/Auth/AdminUser
12:45:37      ✓ should login as authorized master user (5497ms)
12:45:40      ✓ should logout as authorized master user (2265ms)
12:45:40  
12:45:40  
12:45:40    2 passing (9s)
12:45:40  
12:45:40  
12:45:40    (Results)
12:45:40  
12:45:40    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40    │ Tests:        2                                                                                │
12:45:40    │ Passing:      2                                                                                │
12:45:40    │ Failing:      0                                                                                │
12:45:40    │ Pending:      0                                                                                │
12:45:40    │ Skipped:      0                                                                                │
12:45:40    │ Screenshots:  0                                                                                │
12:45:40    │ Video:        false                                                                            │
12:45:40    │ Duration:     8 seconds                                                                        │
12:45:40    │ Spec Ran:     admin_auth.spec.ts                                                               │
12:45:40    └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40  
12:45:40  
12:45:40  resize:  can't open terminal /dev/tty
12:45:40  ====================================================================================================
12:45:40  
12:45:40    (Run Finished)
12:45:40  
12:45:40  
12:45:40         Spec                                              Tests  Passing  Failing  Pending  Skipped  
12:45:40    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40    │ ✔  admin_auth.spec.ts                       00:08        2        2        -        -        - │
12:45:40    └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40      ✔  All specs passed!                        00:08        2        2        -        -        -  
12:45:40  
12:45:40  
12:45:40    Reporting results and artifacts in background...  
12:45:40  
12:45:40   ERROR  TypeError: Cannot read properties of undefined (reading 'flatMap')
12:45:40      at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:27:21
12:45:40      at Array.flatMap (<anonymous>)
12:45:40      at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:26:19
12:45:40      at Array.flatMap (<anonymous>)
12:45:40      at getRunScreenshots (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:25:16)
12:45:40      at getInstanceResultPayload (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:66:18)
12:45:40      at getReportResultsTask (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/uploadResults.ts:26:27)
12:45:40      at createReportTask (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/reportTask.ts:30:5)
12:45:40      at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/runner.ts:58:7
12:45:40      at Array.forEach (<anonymous>)
12:45:40      at runTillDone (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/runner.ts:57:14)
12:45:40  
12:45:40  
12:45:40  ====================================================================================================
12:45:40  
12:45:40  
12:45:40    Cloud Run Finished  
12:45:40  
12:45:40    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40    │     Spec                                         Tests   Passing   Failing   Pending   Skipped │
12:45:40    ├────────────────────────────────────────────────────────────────────────────────────────────────┤
12:45:40    │ ✔   cypress/test/genially/admin/ad      8.6s         2         2         -         -         - │
12:45:40    │     min_auth.spec.ts                                                                           │
12:45:40    ├────────────────────────────────────────────────────────────────────────────────────────────────┤
12:45:40    │ ✔   All specs passed!                   8.6s         2         2         -         -         - │
12:45:40    └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40  
12:45:40  🏁 Recorded Run: https://dashboard-sorrycypress.genially.com/run/4acaa09f2aa7f36f26d5acbec86c675e

The problem is, that the results do not appears on sorry cypress (or the mongodb) and you get 0 tests executed (and the specs runs indefinitely)

[agoldis]

@estefafdez thanks for reporting - will keep cypress-debugger issue separated

@Roemer, @anlambert and @richardmward please check out 1.10.0-beta.3 also, please submit the debug logs when you face issues - would be much easier for us, and thanks for helping with polishing the integration!

[richardmward]

That has sorted my issue. Thank you.

[estefafdez]

@agoldis the debugger issue is already reported there https://github.com/currents-dev/cypress-debugger/issues/59 The problem is that you can run the tests in Cy13 but you can't see the results in sorry cypress, Something seems to be missing as well.

[andreiho]

Anyone else experiencing tests just getting stuck and never finishing? I'm using cypress@13.1.0 with cypress-cloud@1.10.0-beta.3, having followed the migration guide, and the tests start, but it just hangs after the first one. This happens both in open and run mode, and if I remove the cypress-cloud/support and cypress-cloud/plugin parts, the tests work. Any idea what is going on?

[anlambert]

@agoldis I just upgraded to beta 3 and executed the cypress tests of the project I am working on, all good from my side and it indeed fixes the screenshots processing bug, see https://jenkins.softwareheritage.org/blue/organizations/jenkins/DWAPPS%2Fgitlab-builds/detail/gitlab-builds/306/pipeline/88.

Nevertheless, I only use sorry-cypress as tests orchestrator and do not use its reporting dashboards so I might miss other issues.

[estefafdez]

@anlambert what did you see on the cypress dashboard when the run is finished? did you see the results there?

[agoldis]

@estefafdez @richardmward Please check out 1.10.0-beta.4 - it has a few additional fixes.

Also, the corrupted runs could have incompatible data, they should be removed from the DB.

[agoldis]

@andreiho we need more data to explore the case - the debug logs would help

[anlambert]

@estefafdez regarding the sorry-cypress dashboard feature, I bumped cypress-cloud to 1.10.0-beta.4 and tested it locally using that docker-compose.yml file to run the sorry-cypress kit. Everything seems to work as expected, see screenshots below:

[bilarallen]

Anyone else experiencing tests just getting stuck and never finishing? I'm using cypress@13.1.0 with cypress-cloud@1.10.0-beta.3, having followed the migration guide, and the tests start, but it just hangs after the first one. This happens both in open and run mode, and if I remove the cypress-cloud/support and cypress-cloud/plugin parts, the tests work. Any idea what is going on?

@andreiho I also experience this with npx cypress open The test hangs whenever I put import "cypress-cloud/support" in my e2e.js file but if I remove it there will be no problem with cypress runner.

One thing I also noticed is that whenever I check the cypress-cloud/support its index.d.ts file contains an empty export

here's additional info: cypress version: 13.1.0 cypress-cloud version: 1.10.0-beta.4 node version: 20.5.0

[agoldis]

@bilarallen thanks for elaborating on the issue. If that is reproducible, please share a standalone example or and debug logs (https://currents.dev/readme/integration-with-cypress/troubleshooting)

[bilarallen]

@bilarallen thanks for elaborating on the issue. If that is reproducible, please share a standalone example or and debug logs (https://currents.dev/readme/integration-with-cypress/troubleshooting)

Hi @agoldis This can only be replicated when running a test via cypress runner through npx cypress open command, here's my envinfo and I'll also include steps to replicate since this only be replicated when using cypress runner

System:
    OS: Linux 5.10 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (16) x64 AMD Ryzen 7 5800U with Radeon Graphics
    Memory: 4.06 GB / 6.71 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 20.5.0 - ~/.nvm/versions/node/v20.5.0/bin/node
    npm: 9.8.0 - ~/.nvm/versions/node/v20.5.0/bin/npm
  Browsers:
    Chrome: 111.0.5563.64
  npmPackages:
    @babel/eslint-parser: ^7.21.8 => 7.21.8 
    @currents/cli: ^4.0.3 => 4.0.3 
    @cypress/angular:  0.0.0-development 
    @cypress/mount-utils:  0.0.0-development 
    @cypress/react:  0.0.0-development 
    @cypress/react18:  0.0.0-development 
    @cypress/svelte:  0.0.0-development 
    @cypress/vue:  0.0.0-development 
    @cypress/vue2:  0.0.0-development 
    @cypress/webpack-batteries-included-preprocessor: ^3.0.2 => 3.0.2 
    @cypress/webpack-preprocessor: ^5.17.0 => 5.17.0 
    @faker-js/faker: ^7.6.0 => 7.6.0 
    babel-eslint: ^10.1.0 => 10.1.0 
    cy-verify-downloads: ^0.1.16 => 0.1.16 
    cypress: ^13.1.0 => 13.1.0 
    cypress-cloud: ^1.10.0-beta.4 => 1.10.0-beta.4 
    cypress-delete-downloads-folder: 0.0.3 => 0.0.3 
    cypress-network-idle: ^1.14.2 => 1.14.2 
    cypress-real-events: ^1.10.1 => 1.10.1 
    cypress-recurse: ^1.27.0 => 1.27.0 
    eslint: ^8.39.0 => 8.39.0 
    moment: ^2.29.4 => 2.29.4 
    moment-timezone: ^0.5.43 => 0.5.43 
    numeral: ^2.0.6 => 2.0.6 
    swagger-client: ^3.18.5 => 3.18.5 
    uuid: ^9.0.0 => 9.0.0 (8.3.2)
    xvfb: ^0.4.0 => 0.4.0 
  npmGlobalPackages:
    corepack: 0.19.0
    npm: 9.8.0

Steps to replicate when using cypress runner and importing cypress-cloud/support in e2e.js file:

1. Install and use cypress version 12.3.1
2. Install and use cypress-cloud version 1.10.0-beta.4
3. Go to cypress > support > e2e.js file
4. import `cypress-cloud/support` in e2e.js file
5. Run a test using cypress runner via `npx cypress open` command and select electron browser
6. Wait for test execution to finish and observe if it hang
7. Remove import `cypress-cloud/support` in e2e.js file and repeat steps 5 to 6

[henrikvolmer]

@agoldis @bilarallen : I'm experiencing the same issue with cypress open. Is there any solution yet?

[estefafdez]

Ok, for me is finally working with:

 "cypress": "13.1.0",
    "cypress-cloud": "1.10.0-beta.4",
    "cypress-debugger": "1.0.7",

@agoldis thanks!

[andreiho]

@agoldis @bilarallen I've been trying to create a reproducible example, but can't manage to. It appears to be related to some of the custom behaviour we have implemented to deal with some technicalities in our application. But I can't quite explain how importing the cypress-cloud code results in the tests hanging and cypress freezing completely. I'll try to give a bit more context into our situation.

We have a few custom commands which we run on most tests, in order to perform certain actions before proceeding with the test. In this case, the navigateTo command will wait for the page to finish loading, then wait for an element #alerts-holder to be present and attempt to clear any notifications using the clickNotificationButton command in order to prevent notifications covering up content which might need to be clicked in the test.

Now I've tried changing different things in these commands in an attempt to maybe fix some issues, but I can't quite get to the bottom of it, particularly why importing cypress-cloud ends up manifesting this way. It does however work if I simply just never call the clickNotificationButton command, even with cypress-cloud imported. The way the issue manifests is that the tests just freeze, along with the cypress browser, requiring a force quit of the process. It freezes so badly, that it makes the CPU on my M1 Pro go crazy.

This could of course be sub-optimal code on our part, but why would it only manifest itself this way while cypress-cloud is imported? I've tried looking at your plugin and support code, and with my limited knowledge I can't quite tell what could be causing issues there.

Our custom commands

```ts export const navigateTo = ( routePath = '/', options: { isLoggedIn?: boolean; force?: boolean } = {}, ): void => { const url = new URL(`${Cypress.env('BASE_URL')}${routePath}`); const { force = false, isLoggedIn = true } = options; cy.url().then((href) => { if (force || href !== url.href) { cy.visit(url.href); cy.get('body').should('not.have.class', 'loading'); if (isLoggedIn) { cy.get('#alerts-holder', { timeout: 20000 }).should('exist'); cy.clickNotificationButton(); } } }); }; export const clickNotificationButton = ( buttonText?: string, ): Cypress.Chainable> => { return cy .get('#alerts-holder') .as('alerts') .then(($aw) => { if (buttonText) { cy.get('@alerts') .contains(button, buttonText) .as('button') .then(() => { cy.get('@button').click(); }); } else if ($aw.find('[aria-label="close"]').length) { cy.safeClick( () => cy.get('#alerts-holder button[aria-label="close"]'), { multiple: true, }, ); } }); }; export const safeClick = ( cySelectorFn: () => Cypress.Chainable>, clickOpts?: Partial, ): Cypress.Chainable> => { return cy .waitUntil( () => { return cySelectorFn() .as('clickable') .wait(10) // https://github.com/cypress-io/cypress/issues/7306 .then(($el) => Cypress.dom.isAttached($el)); }, { timeout: 1000, interval: 10 }, ) .get('@clickable') .click(clickOpts); }; ```

[andreiho]

Okay after poking around a bit more, I think I may have found the culprit. You can probably disregard most of what I said in my previous message, although I still think there's some relation between the how tests are written, and what happens further down.

The cypress-cloud/support file which you can see here has this piece of code attempting to stringify the entire test object using fast-safe-stringify. Without this safeStringify call, the tests run perfectly. I haven't looked deeper into what exactly happens, but the full test object looks like a fairly complex thing with circular references, which the lib might struggle with (even though it's designed for that), possibly resulting in a recursive loop of sorts and blocking everything else from executing.

cy.task(
  `currents:test:after:run`,
  safeStringify({
    ...test,
    fullTitle: test.fullTitle()
  }),
  {
    log: false
  }
);

Screenshot of the `test` object in my case

[agoldis]

@andreiho thanks a lot for the detailed response. Any chance you can share the test / sample with me somehow? Another approach would be to put a breakpoint right before stringifying the result and make sure that the function returns.

[andreiho]

@agoldis I did try a breakpoint and calling the stringify function in the console, and it never returns, it just hangs indefinitely and the browser becomes virtually unusable, requiring force quit.

Haven't been able to put something useful together to share, as it seems somewhat related to some of the more complex orchestration of our application, with large number of requests made on load and various mocks and intercepts. The test itself is as simple as it gets, I tried running it on a dummy HTML webpage and everything works fine, which makes me think it's something else in the background that is part of the test object which the stringifier doesn't like. I might try manually inspecting that test object before/after commenting out some of the code I previously mentioned which seems to unblock the tests, and see what the differences are.

[agoldis]

@andreiho would you be able to join a quick support call?

[andreiho]

@andreiho would you be able to join a quick support call?

@agoldis I'm not able right now, it's late here in Europe, but I'll try to do a bit more digging tomorrow at work and maybe see if I can somehow give you access to something reproducible.

[andreiho]

@agoldis I went through the test object and tried running safeStringify on each property, to see where it chokes. The 2 things that fail are commands and parent (which circularly references the same test object).

A few examples of things that fail. In these cases I specifically went through all the properties and the stringifier only freezes on these exact properties, there are probably more scenarios however. To me it just seems that the unpredictability and complexity of this test object is maybe too much for the stringifier library, and maybe another solution should be considered.

1. Commands that have the `$el` property referencing a DOM element


2. Intercept commands that use a regex matcher (`hostname` in this case)


Anyhow, I created a super simple repro example, where I load the website of a tool we use internally, which happens to make tons of requests on load. The test itself is nothing but just loading the page and asserting a button is there. Run this test with cypress open and it should freeze (for some reason it works with run, although in my application it doesn't). If you add a breakpoint in the support/index.js file before stringifying and you try to manually stringify the test.commands array, it should freeze and never return. More specifically the last command in this case asserting the button is visible is the one that fails: 55: {subject: jQuery.fn.init(1), $el: jQuery.fn.init(1), _error: undefined, name: 'assert', message: 'expected **<a.button.>** to be **visible**', …}

[agoldis]

@andreiho that's super-helpful! Will release a fix today

[agoldis]

@andreiho @henrikvolmer @bilarallen Please check out 1.10.0-beta.5 - it has a few fixes 🤞🏻

[bilarallen]

@andreiho @henrikvolmer @bilarallen Please check out 1.10.0-beta.5 - it has a few fixes 🤞🏻

It's working now on my end having the import "cypress-cloud/support"; in my e2e.js 👍

[andreiho]

@agoldis Everything working as expected on my end as well, great work! 🙌 Any ETA on when this will land in a proper release?

[agoldis]

@andreiho adding some tests and fixing TS errors - early next week

[estefafdez]

Working on our side as well, thank you very much @agoldis and the team for your hard work and fast speed to resolve this issue! :)

[estefafdez]

One last thing, the images are not being displayed on sorry-cypress dashboard:

I ran the tests with the debug mode , the logs are attached for you to have a look :)

logs.txt

[midleman]

So, like everyone else, had the same problem of tests hanging when running in Cypress GUI mode. This has now been resolved with beta 5, but now tests are not uploading artifacts or properly reporting results in the cloud. In the Output can clearly see the test executed and the last line of output is:

Reporting results and artifacts in background...

And then I see there is a spec level exception:

No results detected for this spec file. That usually happens because of a cypress crash. See the console output for details.

And here's a snippet of a log from at the end of the test run, I dunno if this is helpful:

 Reporting results and artifacts in background...  

  currents:capture resetting captured stdout +48s
  currents:reportTask Creating report task for instanceId XIv19ssXwZ4n +0ms
  currents:state No results detected for "cypress/tests/features/booking/directBillBooking.cy.ts" +48s
  currents:results reporting instance XIv19ssXwZ4n results... +0ms
  currents:runner Getting batched tasks: 1 +48s
  currents:api network request: { method: 'post', url: 'instances/XIv19ssXwZ4n/cy/results', headers: { Accept: 'application/json, text/plain, */*', 'Content-Type': 'application/json', 'x-cypress-request-attempt': 0, 'x-cypress-version': '13.1.0', 'x-ccy-version': '1.10.0-nopeer.0', 'User-Agent': 'cypress-cloud/1.10.0-nopeer.0', 'x-cypress-run-id': '26f2bda37c77f9bd' }, data: { tests: { config: [Object], tests: [Array], hooks: [] }, results: { stats: [Object], reporterStats: {}, exception: 'No results detected for the spec file. That usually happens because of cypress crash. See the console output for details.', video: false, screenshots: [], hasCoverage: false, tests: [Array] } } } +48s
  currents:api network request: { method: 'post', url: 'runs/26f2bda37c77f9bd/cy/instances', headers: { Accept: 'application/json, text/plain, */*', 'Content-Type': 'application/json', 'x-cypress-request-attempt': 0, 'x-cypress-version': '13.1.0', 'x-ccy-version': '1.10.0-nopeer.0', 'User-Agent': 'cypress-cloud/1.10.0-nopeer.0', 'x-cypress-run-id': '26f2bda37c77f9bd' }, data: { runId: '26f2bda37c77f9bd', groupId: '--browser', machineId: 'cZTDY36rfUxf', platform: { osName: 'linux', osVersion: 'Ubuntu - 20.04', osCpus: [Array], osMemory: [Object], browserName: 'Electron', browserVersion: '106.0.5249.51' }, batchSize: 1 } } +0ms
  cypress:cli child event fired { event: 'close', code: 0, signal: null } +12ms

[agoldis]

@midleman @estefafdez @andreiho @bilarallen @henrikvolme @anlambert @Roemer

We've released a few fixes as 2.0.0-beta.0. Would appreciate your help with checking the newest version. It is still released in beta npm channel: npm i cypress-cloud@beta.

Please note that the major version bump and breaking changes:

• dropping support for NodeJS 14 (EOL)
• the TS definition of run function (import { run } from "cypress-cloud") was changed to support multiple versions of cypress runners

[midleman]

@agoldis YESSS, this solves both issues previously reported:

1. hanging on running in GUI mode
2. results not being reported in Currents Dashboard

Thank you!

[bilarallen]

Great! The fix is confirmed working 😉 thanks @agoldis! 🎉

[agoldis]

@estefafdez any luck?

[estefafdez]

@agoldis working also on our side! :) thankssss

[andreiho]

All looking good here as well 👌

[grushetsky]

After upgrading to Cypress 13.2.0 and cypress-cloud 2.0.0-beta.0 the following error started to occur in the CI runner:

CypressError: `cy.task('currents:test:after:run')` failed with the following error:
The 'task' event has not been registered in the setupNodeEvents method. You must register it before using cy.task()
Fix this in your setupNodeEvents method here:
…/cypress.config.ts
https://on.cypress.io/api/task
Because this error occurred during a `after each` hook we are skipping all of the remaining tests.

The cloud plugin is called in the same way as it was previously called in cypress.config.ts:

export default defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      cloudPlugin(on, config);

      on('before:browser:launch', (browser, launchOptions) => {
        …
      });

      on(
        'file:preprocessor',
        webpackPreprocessor({
          …
        })
      );
    },
  },
});

What is the catch here?

[midleman]

Slight update. Things are working, but I have noticed that we are only getting video uploads in Currents Dashboard if the test fails and not on retries. Historically this has worked fine using their recommended configuration. Just curious if anyone else has the same issue or if I need to look further into my configuration.

[estefafdez]

@midleman in Cypress 13 they removed the video, so no video should be ever sent into the sorry cypress dashboard. Have you tried to do the migration properly? https://www.cypress.io/blog/2023/09/15/announcing-cypress-13-test-replay/

[midleman]

@midleman in Cypress 13 they removed the video, so no video should be ever sent into the sorry cypress dashboard. Have you tried to do the migration properly? https://www.cypress.io/blog/2023/09/15/announcing-cypress-13-test-replay/

I followed the migration guide, which states you can manually set video: true (which I have) - see: here. The videos upload fine to Currents if the test fails, just not on retries. I confirmed the logic in the after:spec is correct as well, so I'm confused why it's not uploading in that particular case. 🤔

[anlambert]

@agoldis Still all good from my side using cypress-cloud 2.0.0-beta.0 and cypress 13.2.0 https://jenkins.softwareheritage.org/blue/organizations/jenkins/DWAPPS%2Fgitlab-builds/detail/gitlab-builds/315/pipeline/88

[mirobo]

@midleman in Cypress 13 they removed the video, so no video should be ever sent into the sorry cypress dashboard. Have you tried to do the migration properly? https://www.cypress.io/blog/2023/09/15/announcing-cypress-13-test-replay/

I followed the migration guide, which states you can manually set video: true (which I have) - see: here. The videos upload fine to Currents if the test fails, just not on retries. I confirmed the logic in the after:spec is correct as well, so I'm confused why it's not uploading in that particular case. 🤔

I have the same issue, but I wasn't able to reproduce it. I created flaky tests with retries, failing tests, all with different nestings and videos were always uploaded on retries and failures. But then in "reality" (with the same code in "after:spec" hook) I sometimes (!) miss videos of retried tests. I didn't investigate further. This happend with Cypress 12,x; I haven't tried 13+ yet..

[agoldis]

@midleman @mirobo @estefafdez

• setting video:true enables videos in cypress 13
• please tell me if setting videoUploadOnPasses: true in cypress.config.js (despite the warnings) solves the issue for you

EDIT: I have updated the guide for setting up cypress-cloud together with multiple plugins. There are a few caveats, please take a look here: https://github.com/currents-dev/cypress-cloud#setup-with-existing-plugins