search

cypress-io / cypress-grep

Filter tests using substring
137 stars 19 forks source link

Burning high number of specs/tests will result in Cypress throwing SIGSEGV error #105

Closed alexniculae closed 2 years ago

alexniculae commented 2 years ago

Running grepBurn = 5 (or more) on a large number of tests (>~20 spec file || >~50 tests) will result in Cypress throwing a SIGSEGV error.

Unfortunately I cannot share the project I am working on, but this should be easy to reproduce with any bigger project that contains more complex tests, if you choose to run all the spec files and burn 5 or even 10 times to make sure.

When the failure occurs, it's always in the last spec file, after the second or third test is ran 2 or 3 times

If I run the same large number of specs/tests with grepBurn = 4, the run will successfully finish.

Console error:

<--- Last few GCs --->
[226:0x318b00000000]  5280146 ms: Mark-sweep 1707.4 (1988.7) -> 1701.4 (1990.2) MB, 344.7 / 0.1 ms  (average mu = 0.843, current mu = 0.807) task scavenge might not succeed
[226:0x318b00000000]  5282173 ms: Mark-sweep 1708.9 (1990.2) -> 1702.9 (1992.0) MB, 390.8 / 0.1 ms  (average mu = 0.825, current mu = 0.807) task scavenge might not succeed
<--- JS stacktrace --->
Error: The Test Runner unexpectedly exited via a exit event with signal SIGSEGV
Please search Cypress documentation for possible solutions:
https://on.cypress.io
Check if there is a GitHub issue describing this crash:
https://github.com/cypress-io/cypress/issues
Consider opening a new issue.
----------
Platform: linux (Debian - 10.9)
Cypress Version: 8.3.0
    at /builds/web/cypressapp/node_modules/cypress/lib/errors.js:257:17
    at tryCatcher (/builds/web/cypressapp/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:729:18)
    at Promise._fulfill (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:673:18)
    at Promise._resolveCallback (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:466:57)
    at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:559:17)
    at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:729:18)
    at Promise._fulfill (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:673:18)
    at Promise._resolveCallback (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:466:57)
    at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:559:17)
    at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18) {
  known: true
}
bahmutov commented 2 years ago

Have you tried using a larger machine?

Sent from my iPhone

On Jan 17, 2022, at 09:44, testingJava2 @.***> wrote:

 Running grepBurn = 5 (or more) on a large number of tests (>~20 spec file || >~50 tests) will result in Cypress throwing a SIGSEGV error.

Unfortunately I cannot share the project I am working on, but this should be easy to reproduce with any bigger project that contains more complex tests, if you choose to run all the spec files and burn 5 or even 10 times to make sure.

When the failure occurs, it's always in the last spec file, after the second or third test is ran 2 or 3 times

if I run the same test in a more limited group of specs/tests with grepBurn =5, the SIGSEGV will not occur If I run the same large number of specs/tests with grepBurn = 4, the run will successfully finish.

Console error:

<--- Last few GCs ---> [226:0x318b00000000] 5280146 ms: Mark-sweep 1707.4 (1988.7) -> 1701.4 (1990.2) MB, 344.7 / 0.1 ms (average mu = 0.843, current mu = 0.807) task scavenge might not succeed [226:0x318b00000000] 5282173 ms: Mark-sweep 1708.9 (1990.2) -> 1702.9 (1992.0) MB, 390.8 / 0.1 ms (average mu = 0.825, current mu = 0.807) task scavenge might not succeed <--- JS stacktrace ---> Error: The Test Runner unexpectedly exited via a exit event with signal SIGSEGV Please search Cypress documentation for possible solutions: https://on.cypress.io Check if there is a GitHub issue describing this crash: https://github.com/cypress-io/cypress/issues Consider opening a new issue.

Platform: linux (Debian - 10.9) Cypress Version: 8.3.0 at /builds/web/cypressapp/node_modules/cypress/lib/errors.js:257:17 at tryCatcher (/builds/web/cypressapp/node_modules/bluebird/js/release/util.js:16:23) at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:547:31) at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18) at Promise._settlePromise0 (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:649:10) at Promise._settlePromises (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:729:18) at Promise._fulfill (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:673:18) at Promise._resolveCallback (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:466:57) at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:559:17) at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18) at Promise._settlePromise0 (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:649:10) at Promise._settlePromises (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:729:18) at Promise._fulfill (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:673:18) at Promise._resolveCallback (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:466:57) at Promise._settlePromiseFromHandler (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:559:17) at Promise._settlePromise (/builds/web/cypressapp/node_modules/bluebird/js/release/promise.js:604:18) { known: true } — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

alexniculae commented 2 years ago

@bahmutov, I have upped the RAM of my VM from 4gb to 6 and have ran some tests, but the SIGSEGV is still occurring.

For reproducing this: I am doing some tests to run ~30 spec files X ~3 tests each X 5 burns and usually the crash happens towards the end of the run.

When running with 4 burns or less this crash does not occur.

alexniculae commented 2 years ago

@bahmutov, could you please let me know what machine you would recommend for running a high number of Cy tests? Should one of 4 or 6 gb RAM do a good job? (I don't see why more then that would be required, but maybe I am missing something 🙂 )

bahmutov commented 2 years ago

@alexniculae this is your application and your tests, I cannot tell how big of a machine needs to be to run it. Please try a bigger machine, or use a different browser to run the tests, or run the app in a different container to lessen the memory requirement.

alexniculae commented 2 years ago

Bigger machine solved the issue - it is indeed a quite stressful job I was asking Cy to do

Closing the issue as, after reading about Cypress memory consumption this is expected to happen under such stress

Thank you Gleb 🙂