cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.02k stars 3.18k forks source link

Cypress hangs while loading a test with large code base #29059

Closed iomedico-beyer closed 1 month ago

iomedico-beyer commented 8 months ago

Current behavior

In my test I reference a large code base I have in /cypress/support. After reaching a certain ammount of code (currently 2.3 million lines), I noticed Chrome, Edge and Electron would hang forever (I stopped them after an hour) while loading the test. My machine did not run out of memory (total 85.7 GB, free 43.1 GB). Firefox also takes 2 minutes, but then at least runs the test. But as I need test videos, Firefox is not an option.

Those files are TypeScript files, but transpiling them to JavaScript first (using SWC) did not help.

I tried Cypress 12.17.3, because it still had webpack 4. There Electron and Chrome also hang but at least gave me a stacktrace. First line: FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

I tried to increase RAM like this $env:NODE_OPTIONS="--max-old-space-size=16384" but that did not help either. Also then starting Cypress warns me that “Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.”

Edit: After further research I discovered that Chromium based browsers have a 4 GB heap limit per tab. The debug log says Chrome consumes 4392.89 MB. Could this be the real problem I am facing?

Desired behavior

  1. Please don't hang
  2. Please provide a way to configure the available RAM.
  3. Please provide a way to separate webpack compilation from the test run. The former takes 93 seconds in my case (see below).

Test code to reproduce

-

Cypress Version

13.6.6

Node version

16.16.0

Operating System

Windows 10.0.19045

Debug Logs

...
webpack 5.88.2 compiled successfully in 93049 ms
...
  cypress:server:util:process_profiler ┌─────────┬───────────┬──────────────┬──────────────────────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │   group   │ processCount │                       pids                       │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────┼──────────────┼──────────────────────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │ 'cypress' │      1       │                     '910452'                     │    0.27    │      0.79      │  332.35  │    330.24    │   352.09    │
  cypress:server:util:process_profiler │    1    │ 'Chrome'  │      5       │     '911596, 911920, 916324, 903352, 891312'     │     0      │      4.62      │  294.05  │    693.13    │   4392.89   │
  cypress:server:util:process_profiler │    2    │  'other'  │      6       │ '913548, 913500, 912028, 909860, 887256, 906600' │     0      │       0        │  248.79  │    248.08    │   249.11    │
  cypress:server:util:process_profiler │    3    │ 'plugin'  │      2       │                 '894860, 915036'                 │     0      │      7.55      │  153.96  │    529.17    │   2814.09   │
  cypress:server:util:process_profiler │    4    │ 'ffmpeg'  │      1       │                     '914604'                     │     0      │      0.36      │  66.85   │    63.76     │    66.88    │
  cypress:server:util:process_profiler │    5    │  'TOTAL'  │      15      │                       '-'                        │    0.27    │     13.05      │ 1096.01  │   1824.54    │   5188.7    │
  cypress:server:util:process_profiler └─────────┴───────────┴──────────────┴──────────────────────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +12s
  cypress:https-proxy received error on client browserSocket { err: Error: read ECONNRESET     at TCP.onStreamRead (node:internal/stream_base_commons:217:20) { errno: -4077, code: 'ECONNRESET', syscall: 'read' }, url: 'foo:443' } +14s
  cypress:server:util:process_profiler current & mean memory and CPU usage by process group:
  cypress:server:util:process_profiler ┌─────────┬───────────┬──────────────┬──────────────────────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │   group   │ processCount │                       pids                       │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────┼──────────────┼──────────────────────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │ 'cypress' │      1       │                     '910452'                     │    0.31    │      0.76      │  333.95  │    330.43    │   352.09    │
  cypress:server:util:process_profiler │    1    │ 'Chrome'  │      5       │     '911596, 911920, 916324, 903352, 891312'     │     0      │      4.37      │  294.06  │    672.12    │   4392.89   │
  cypress:server:util:process_profiler │    2    │  'other'  │      6       │ '913548, 913500, 912028, 909860, 913292, 911012' │     0      │       0        │  248.68  │    248.11    │   249.11    │
  cypress:server:util:process_profiler │    3    │ 'plugin'  │      2       │                 '894860, 915036'                 │     0      │      7.17      │  153.96  │    510.41    │   2814.09   │
  cypress:server:util:process_profiler │    4    │ 'ffmpeg'  │      1       │                     '914604'                     │     0      │      0.34      │  66.85   │    63.92     │    66.88    │
  cypress:server:util:process_profiler │    5    │  'TOTAL'  │      15      │                       '-'                        │    0.31    │     12.41      │ 1097.51  │   1788.19    │   5188.7    │
  cypress:server:util:process_profiler └─────────┴───────────┴──────────────┴──────────────────────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +12s

Other

-

jennifer-shehane commented 8 months ago

Could you explain this in a bit more detail?

In my test I reference a large code base I have in /cypress/support

Do you have a require or import or something to your entire code base within support? Why? Or do you have code files in there? Or is this just test files?

iomedico-beyer commented 8 months ago

Those files in /cypress/support are a domain specific API on top of Cypress to test a very complex product of ours (in fact a set of producs). Each file is a page objects (-> design pattern). They contain many calls to Cypress, but no test cases.

A test imports only one of those page objects bot which is kind of an entry point, but bot imports all the other. This way in their IDE (Visual Studio Code), the test writers can navigate through all the page objects just by adding call after call, always getting the correct code completion.

Usage example with additional comments (for you) to indicate when a different page object is returned:

   import { bot } from "foostudy"

   it("creates a patient record", function() {
      bot.loginTestuser().asInvestigator() // returns bot
         .clickCreatePatient() // returns FD_INFORMEDCONSENT
         .getID_DS_DSICDAT4().enter("22.02.2022")
         .getID_DM_BRTHYR().enter("1955") 
         .getID_XX_TISSUESENDFDCONSYN().selectNo()
         .clickSaveAndNext().isFD_INCLUSIONEXCLUSION() // returns FD_INCLUSIONEXCLUSION
         .getID_QS_QSECOG().selectGrade0FullyActiveAbleToCarryOnAllPreDiseasePerformance() 
         // etc.
   })

   it("fills out Baseline: Disease characteristics at primary diagnosis", function() {
      bot.loginTestuser().asInvestigator()
         .selectLastPatient() // returns bot
         .visitSE_BASELINE() // returns SE_BASELINE
         .visitFD_DISEASECHARPRIM() // returns FD_DISEASECHARPRIM
         .getID_TU_PRIMDIAGTULOC().selectBilateral()
         // etc.
   })

The API is generated from CDISC ODM files, but I guess that doesn't matter.

iomedico-beyer commented 8 months ago

After updating node from v16.16.0 to v21.7.0 I see something different:

  cypress:server:util:process_profiler ┌─────────┬───────────┬──────────────┬───────────────────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │   group   │ processCount │                     pids                      │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────┼──────────────┼───────────────────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │ 'cypress' │      1       │                   '1035904'                   │    0.25    │      0.46      │  336.12  │    334.17    │   354.23    │
  cypress:server:util:process_profiler │    1    │ 'Chrome'  │      5       │ '1025912, 1013444, 1041284, 1041384, 1031088' │     0      │      1.13      │  311.94  │    446.93    │   3429.13   │
  cypress:server:util:process_profiler │    2    │  'other'  │      5       │ '1014880, 1037836, 1025756, 1033300, 1041940' │     0      │       0        │  204.29  │    203.95    │   204.48    │
  cypress:server:util:process_profiler │    3    │ 'plugin'  │      2       │               '895040, 924152'                │     0      │      2.37      │  154.45  │    553.69    │   3518.32   │
  cypress:server:util:process_profiler │    4    │ 'ffmpeg'  │      1       │                   '1007316'                   │     0      │      0.14      │  66.98   │    65.68     │    67.01    │
  cypress:server:util:process_profiler │    5    │  'TOTAL'  │      14      │                      '-'                      │    0.25    │      4.07      │ 1073.78  │   1592.77    │   7545.47   │
  cypress:server:util:process_profiler └─────────┴───────────┴──────────────┴───────────────────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +13s

So now it's only about 3.4gb used for Chrome, but it still hangs. Strange.

But it has to do with the size, because I can use two specific parts of the API separately, but not together.

iomedico-beyer commented 7 months ago

Also interesting: The Chrome tab crashes but Cypress doesn't notice. Normally it is 6 processes. In the logs (see above) there are only 5 processes left ...

cypress-app-bot commented 1 month ago

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

cypress-app-bot commented 1 month ago

This issue has been closed due to inactivity.