"Out of memory crash" message is displayed on Chrome of Cypress

[mauceballos]

Description: When We run Github actions the step to run the test suite in cypress fails, and the logs showed an error after each and before each hook. When we run the test suite locally, the execution fails too, but We can see the error when turn-on the dev tool on the browser. the error message that shows on the dev tool is "out of memory crash". (Error code: SIGTRAP cypress chrome)

Workaround:

• The first possible solution we came up with was to split the test case that consumed over 4000 megabytes of memory load. the execution after the changes continued to fail and showing an "out of memory crash" message.

• Another possible solution was to optimize the scenario that continuing failed, was reduce the lines or steps and the BDD because this has more than 8 lines, the exedent lines move to the background

• Another possible solution was to optimize the scenario that kept failing, reducing the lines or steps in the BDD scenario because it has more than 8 or more lines. these lines would go to the background step and the feature would stay with only When and Then. after this change continued to fail and showing an "out of memory crash" message.

• Another possible solution was to optimize the hook was including cy.clearLocalStorage() and cy.clearCookies() on afterEach(), this change stabilized the execution of the test suite but did not fully resolve the issue.

console log lastest execution:

 Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  features/filters/add-filter-from-ag      03:48        6        -        6        -        - │
  │    ent-events.feature                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-filter-from-ag      03:06        5        -        5        -        - │
  │    ent-events2.feature                                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-filter-from-ag      04:57       11       10        1        -        - │
  │    ent.feature                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-new-filter-fro      03:33       11       10        1        -        - │
  │    m-events.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-new-filter.fea      02:58       11       10        1        -        - │
  │    ture                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-da      03:44       11       10        1        -        - │
  │    shboard-to-events.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-ev      04:10       11       10        1        -        - │
  │    ents-to-events.feature                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter.feature      03:05       11       10        1        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/remove-filter-from      05:03       22       21        1        -        - │
  │    -module-dashboard.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/decoders/should      00:44        3        3        -        -        - │
  │    -list-decoders-and-show-the-pager.f                                                         │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/rules/should-li      00:35        2        2        -        -        - │
  │    st-rules-and-show-the-pager.feature                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/about.      00:30        1        1        -        -        - │
  │    feature                                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/api-co      00:48        3        3        -        -        - │
  │    nfigurations.feature                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/config      01:24        1        1        -        -        - │
  │    uration.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/disabl      02:49       11       11        -        -        - │
  │    e-modules.feature                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/enable      02:54       11       11        -        -        - │
  │    -modules.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/logs.f      00:39        2        2        -        -        - │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/miscel      00:35        1        1        -        -        - │
  │    laneous.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/sample      01:32        2        2        -        -        - │
  │    -data.feature                                                                               │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  9 of 19 failed (47%)                     47:03      136      118       18        -        -  

researching the error "out of memory crash" message

We regularly encountered out-of-memory issues with Cypress too - and found a solution in our case: The deeper problem should be fixed in Cypress itself, but we also found a user workaround.

• Problem:

  When running Cypress test in headed mode, we regularly got crashes with out of memory. numTestsKeptInMemory doesn't help, because a single test alone is enough to crash it. When did it happen: The waste of memory always exist, but becomes critical in applications where you have... Several XHR requests (>30 in our case) and/or A complex UI with large CSS files.

• Reason:

  Main memory issue is caused by the headStyles stored in each snapshot: For any small step (request/response/whatever else triggers snapshots) the whole set of headStyles is collected again and again - although there are often no or very little changes. User-Workaround: In our case, the problem was particularly heavy due to some quite long .css-files. By cleaning them up, we could signficiantly reduce the memory consumption. E.g., when using scss import without deduplication in dev mode, a style file can easily get much bigger than necessary. There may be embedded base-64 images etc. - any of this is duplicated hundreds of time by Cypress, so that reducing the .css files can help a lot. How could it be fixed in Cypress: To verify my theory, I did a local proof-of-concept hack directly in the cypress_runner.js bundle file: I added a simple string dictionary that reduced the memory usage by more than a Gigabyte in our case (>50% of the overall heap size) - which fully fixed the problem without losing functionality. We hadn't cleaned our css files at that point though, so the headStyles were about 3MB - so the gain may be less spectacular in other scenarios.

It worked as follows: Instead of storing strings in the headStyles array, I just store an array of indices into a shared string dictionary. When the styles are used (in function replaceHeadStyles), I unpack the styles, i.e., replace the indices back into an array of strings. To make sure that the strings are not leaked in the Dictionary, Log.reduceMemory() should unregister the strings before deleting any snapshots. I ignored that in the experimental hack though, because the dictionary size never got critical anyway. I am aware that a proper fix needs to happen in the original coffee scripts, but it was just for trying it out locally and for making my suggestion more concrete here.

_Step 1: In the begging of cypressrunner.js, I added the global string dictionary:

function StringMap() {

      this.stringToIndex = new Map();
      this.indexToString = [];

      this.addString = function(str) {

      // If this string is known, just return its index
      var index = this.stringToIndex.get(str);
      if (index !== undefined) {
          return index;
      }

      // This is a new string. Add it to both maps.
      index = this.indexToString.length;
      this.indexToString.push(str);
      this.stringToIndex.set(str, index);
      return index;
  };

  this.getString = function(index) {
      return this.indexToString[index];
  };

  this.addStrings = function(strArray) {
      var indices = [];
      for (var i=0; i<strArray.length; i++) {
          var s = strArray[i];
          var index = this.addString(s);
          indices.push(index);
      }
      return indices;
  };

  this.getStrings = function(indices) {
      var strArray = [];
      for (var i=0; i<indices.length; i++) {
          var index = indices[i];
          var str   = this.getString(index);
          strArray.push(str);
      }
      return strArray;
  };
};
window.stringMap = new StringMap();

// Adds an array of style strings to the dictionary.
// Returns an array of indices into the global string dictionary.
window.packStyles = function(styles) {
    return window.stringMap.addStrings(styles);
};

// Gets an array of indices into the string dictionary pointing to style strings.
// Returns an array containing the original strings.
window.unpackStyles = function(styles) {

    // Return identiy if styles is undefined or already unpacked
    var isPacked = styles && (typeof styles[0] == "number");
    if (!isPacked) {
      return styles;
    }
    return window.stringMap.getStrings(styles);
};

Step 2: Add this to createSnapshot() (right after getting headStyles and bodyStyles)

// replace array of style-strings by array of string-ids
headStyles = window.stringMap.addStrings(headStyles);

Step 3: Add this to _replaceHeadStyles (at the beginning) styles = window.unpackStyles(styles); Let me know if there are any questions I can help with.

_About Performance: I did some profiling and verified that my dictionary approach didn't have significant impact on the overall performance. Alternative/Better fixes: The approach described above is just one possible solution. Profiling also showed that the getStylesFor() function took nearly 30% of the time in our tests. So, even better than avoiding double-storage would be a way to reduce the work to collect the styles. I didn't investigate this in detail though, e.g., I am not sure if/how it is possible to track changes in the headstyles. But I think even just getting rid of the memory duplication would already help a lot of users to get rid of memory overflow issues. An even simpler option to provide that would be a more fine-granular control to decide for which steps Cypress takes snapshots. The only way to skip snapshots selectively from outside is the whitelist in the cy.server config - but since it does more than just skipping the snapshots, it didn't help here either.

[mauceballos]

We will include a new parameter on the cypress.json "numTestsKeptInMemory": 0, to try to solve the issue, also we will roll back the changes to the features. to know how the execution behaves.

execution with --headless chrome Console Log:

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  features/filters/add-filter-from-ag      07:39       11        -       11        -        - │
  │    ent-events.feature                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-filter-from-ag      06:33       11       10        1        -        - │
  │    ent.feature                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/filters/add-new-filter-fro      03:39       11       11        -        -        - │
  │    m-events.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-new-filter.fea      04:04       11        9        2        -        - │
  │    ture                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-da      04:29       11       10        1        -        - │
  │    shboard-to-events.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-ev      04:31       11       10        1        -        - │
  │    ents-to-events.feature                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter.feature      04:18       11        9        2        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/remove-filter-from      05:05       22       20        2        -        - │
  │    -module-dashboard.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/decoders/should      00:40        3        3        -        -        - │
  │    -list-decoders-and-show-the-pager.f                                                         │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/rules/should-li      00:34        2        2        -        -        - │
  │    st-rules-and-show-the-pager.feature                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/about.      00:29        1        1        -        -        - │
  │    feature                                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/api-co      00:45        3        3        -        -        - │
  │    nfigurations.feature                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/config      01:06        1        1        -        -        - │
  │    uration.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/management/settings/disabl      02:37       11       10        1        -        - │
  │    e-modules.feature                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/management/settings/enable      02:55       11       10        1        -        - │
  │    -modules.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/logs.f      00:37        2        2        -        -        - │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/miscel      00:29        1        1        -        -        - │
  │    laneous.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/sample      01:25        2        2        -        -        - │
  │    -data.feature                                                                               │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  9 of 18 failed (50%)                     52:04      136      114       22        -        -  

[mauceballos]

execution with --headed --browser chrome

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  features/filters/add-filter-from-ag      07:47       11        -       11        -        - │
  │    ent-events.feature                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-filter-from-ag      07:10       11        4        7        -        - │
  │    ent.feature                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-new-filter-fro      04:47       11        5        6        -        - │
  │    m-events.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/add-new-filter.fea      04:03       11        7        4        -        - │
  │    ture                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-da      05:44       11        7        4        -        - │
  │    shboard-to-events.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter-from-ev      06:09       11        8        3        -        - │
  │    ents-to-events.feature                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/pin-filter.feature      05:54       11        4        7        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  features/filters/remove-filter-from      11:11       22        7       15        -        - │
  │    -module-dashboard.feature                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/decoders/should      00:47        3        3        -        -        - │
  │    -list-decoders-and-show-the-pager.f                                                         │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/rules/should-li      00:37        2        2        -        -        - │
  │    st-rules-and-show-the-pager.feature                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/about.      00:32        1        1        -        -        - │
  │    feature                                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/api-co      00:52        3        3        -        -        - │
  │    nfigurations.feature                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/config      01:14        1        1        -        -        - │
  │    uration.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/disabl      02:47       11       11        -        -        - │
  │    e-modules.feature                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/enable      02:51       11       11        -        -        - │
  │    -modules.feature                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/logs.f      00:43        2        2        -        -        - │
  │    eature                                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/miscel      00:32        1        1        -        -        - │
  │    laneous.feature                                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  features/management/settings/sample      01:32        2        2        -        -        - │
  │    -data.feature                                                                               │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  8 of 18 failed (44%)                   1:05:21      136       79       57        -        -  

[Mayons95]

Merged!