search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.7k stars 3.16k forks source link

Cypress & Chrome processes do not finish #17207

Closed mpressmar closed 3 years ago

mpressmar commented 3 years ago

We have a frequently appearing issue in our CI environment (GitLab with docker runners) where the pipeline never finishes running. Test execution effectively takes only a few minutes, but the Cypress and Chrome processes do not terminate - until after 2 hours the Pipeline's time limit is reached and they are forcefully killed.

Versions used:

The pipeline log:

$ time npm ci --cache .npm --prefer-offline
> fsevents@1.2.13 install /builds/d-rd/web-e2e/node_modules/webpack-dev-server/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> fsevents@1.2.13 install /builds/d-rd/web-e2e/node_modules/watchpack-chokidar2/node_modules/fsevents
> node install.js
Skipping 'fsevents' build as platform linux is not supported
> core-js@3.6.4 postinstall /builds/d-rd/web-e2e/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
> sonar-scanner@3.1.0 postinstall /builds/d-rd/web-e2e/node_modules/sonar-scanner
> node postinstall.js
> cypress@7.2.0 postinstall /builds/d-rd/web-e2e/node_modules/cypress
> node index.js --exec install
[03:23:59]  Downloading Cypress     [started]
[03:24:10]  Downloading Cypress     [completed]
[03:24:10]  Unzipping Cypress       [started]
[03:24:32]  Unzipping Cypress       [completed]
[03:24:32]  Finishing Installation  [started]
[03:24:32]  Finishing Installation  [completed]
> @angular/cli@10.2.3 postinstall /builds/d-rd/web-e2e/node_modules/@angular/cli
> node ./bin/postinstall/script.js
added 1687 packages in 121.084s
real    2m2.156s
user    1m14.216s
sys 0m55.204s
$ npx cypress cache path
/builds/d-rd/web-e2e/cache/Cypress
$ npx cypress cache list
┌─────────┬──────────────┐
│ version │ last used    │
├─────────┼──────────────┤
│ 7.2.0   │ 6 months ago │
└─────────┴──────────────┘
$ time npm run cypress:verify
> esafe-e2e@0.0.0 cypress:verify /builds/d-rd/web-e2e
> npm run build:api-client && cypress verify
> esafe-e2e@0.0.0 build:api-client /builds/d-rd/web-e2e
> tsc --project src/api-client/tsconfig.json
⚠ Warning: Binary version 6.3.0 does not match the expected package version 7.2.0
  These versions may not work properly together.
[03:24:41]  Verifying Cypress can run /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress [started]
[03:24:46]  Verifying Cypress can run /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress [completed]
real    0m10.658s
user    0m10.440s
sys 0m1.824s

> cypress run --config-file cypress.json "--headless" "--browser" "chrome" "--config" "video=false"
??? Warning: Binary version 6.3.0 does not match the expected package version 7.2.0
  These versions may not work properly together.
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
====================================================================================================
  (Run Starting)
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
  ??? Cypress:      6.3.0                                                                            ???
  ??? Browser:      Chrome 87 (headless)                                                             ???
  ??? Specs:        1 found (common/dummy-test.common.spec.ts)                                       ???
  ??? Experiments:  experimentalStudio=true                                                          ???
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

  Running:  common/dummy-test.common.spec.ts                                                (1 of 1)
[cypress-log-to-output] Attempting to connect to Chrome Debugging Protocol
[cypress-log-to-output] Connected to Chrome Debugging Protocol
Timed out waiting for the browser to connect. Retrying...
[cypress-log-to-output] Attempting to connect to Chrome Debugging Protocol
Warning: Cypress failed to connect to Chrome via stdio after 1 minute, 0 seconds. Falling back to TCP...
Connecting to Chrome via TCP was successful, continuing with tests.
[cypress-log-to-output] Connected to Chrome Debugging Protocol
[2021-07-06T01:28:08.169Z]   console.clear called
                             Arguments:
                               [
                                 {
                                   "type": "string",
                                   "value": "console.clear"
                                 }
                               ]
[2021-07-06T01:28:12.676Z]   console.clear called
                             Arguments:
                               [
                                 {
                                   "type": "string",
                                   "value": "console.clear"
                                 }
                               ]
  (Results)
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
  ??? Tests:        1                                                                                ???
  ??? Passing:      1                                                                                ???
  ??? Failing:      0                                                                                ???
  ??? Pending:      0                                                                                ???
  ??? Skipped:      0                                                                                ???
  ??? Screenshots:  0                                                                                ???
  ??? Video:        false                                                                            ???
  ??? Duration:     0 seconds                                                                        ???
  ??? Spec Ran:     common/dummy-test.common.spec.ts                                                 ???
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
[cypress-log-to-output] Chrome Debugging Protocol disconnected
====================================================================================================
  (Run Finished)
       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
  ??? ???  common/dummy-test.common.spec.ts         176ms        1        1        -        -        - ???
  ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
    ???  All specs passed!                        176ms        1        1        -        -        -  
ERROR: Job failed: execution took longer than 2h0m0s seconds

The issue may appear whether all tests pass or fail. It also appears with a very simple dummy test, such as this:

describe('Dummy-Test', () => {
  it('should always pass', () => {});
});

The processes that remain on the docker appear as follows:

  PID TTY      STAT   TIME COMMAND
  626 ?        S      0:00 /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress/Cypress --type=zygote --no-zygote-sandbox --no-sandbox
  627 ?        S      0:00 /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress/Cypress --type=zygote --no-sandbox
  639 ?        S      0:00 /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress/Cypress --type=zygote --no-zygote-sandbox --no-sandbox
  640 ?        S      0:00 /builds/d-rd/web-e2e/cache/Cypress/7.2.0/Cypress/Cypress --type=zygote --no-sandbox
 1148 ?        Sl     0:01 /opt/google/chrome/chrome --test-type --ignore-certificate-errors --start-maximized --silent-debugger-extension-api --no-default-browser-check --no-first-run --noerrdialogs --enable-fixed-layout --disable-popup-blocking --disable-password-generation --disable-single-click-autofill --disable-prompt-on-repos --disable-background-timer-throttling --disable-renderer-backgrounding --disable-renderer-throttling --disable-backgrounding-occluded-windows --disable-restore-session-state --disable-new-profile-management --disable-new-avatar-menu --allow-insecure-localhost --reduce-security-for-testing --enable-automation --disable-device-discovery-notifications --autoplay-policy=no-user-gesture-required --disable-site-isolation-trials --metrics-recording-only --disable-prompt-on-repost --disable-hang-monitor --disable-sync --disable-web-resources --safebrowsing-disable-download-protection --disable-client-side-phishing-detection --disable-component-update --disable-default-apps --use-fake-ui-for-media-stream --use-fake-device-for-media-stream --disable-ipc-flooding-protection --disable-backgrounding-occluded-window --disable-breakpad --password-store=basic --use-mock-keychain --disable-dev-shm-usage --disable-gpu --no-sandbox --proxy-server=http://localhost:41867 --proxy-bypass-list=<-loopback> --headless --window-size=1280,720 --remote-debugging-port=40245 --remote-debugging-address=127.0.0.1 --remote-debugging-pipe --user-data-dir=/root/.config/Cypress/cy/production/browsers/chrome-stable/run-637 --disk-cache-dir=/root/.config/Cypress/cy/production/browsers/chrome-stable/run-637/CypressCache about:blank
 1159 ?        S      0:00  \_ cat
 1160 ?        Z      0:00  \_ [cat] <defunct>
 1163 ?        S      0:00  \_ /opt/google/chrome/chrome --type=zygote --no-zygote-sandbox --no-sandbox --headless --headless
 1182 ?        Sl    77:09  |   \_ /opt/google/chrome/chrome --type=gpu-process --field-trial-handle=11660582303243077558,4307428391452619947,131072 --no-sandbox --disable-dev-shm-usage --disable-breakpad --headless --ozone-platform=headless --headless --gpu-preferences=OAAAAAAAAAAgAAAQAAAAAAAAAAAAAAAAAABgAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAA== --use-gl=swiftshader-webgl --override-use-software-gl-for-tests --shared-files
 1164 ?        S      0:00  \_ /opt/google/chrome/chrome --type=zygote --no-sandbox --headless --headless
 1186 ?        Sl     0:51  |   \_ /opt/google/chrome/chrome --type=renderer --no-sandbox --disable-dev-shm-usage --autoplay-policy=no-user-gesture-required --disable-background-timer-throttling --disable-breakpad --enable-automation --remote-debugging-pipe --remote-debugging-port=40245 --test-type --use-fake-ui-for-media-stream --allow-pre-commit-input --ozone-platform=headless --field-trial-handle=11660582303243077558,4307428391452619947,131072 --disable-gpu-compositing --lang=en-US --headless --lang=en-US --num-raster-threads=4 --enable-main-frame-before-activation --renderer-client-id=4 --shared-files=v8_context_snapshot_data:100
 1183 ?        Sl     0:01  \_ /opt/google/chrome/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --field-trial-handle=11660582303243077558,4307428391452619947,131072 --lang=en-US --service-sandbox-type=network --no-sandbox --disable-dev-shm-usage --ignore-certificate-errors --proxy-server=http://localhost:41867 --use-fake-device-for-media-stream --use-gl=swiftshader-webgl --ignore-certificate-errors --headless --shared-files=v8_context_snapshot_data:100

We run the tests as: cypress run --config-file cypress.json

Our cypress.json:

{
  "_comment": "Cypress validates that baseUrl is valid, but it will be overwritten by env.baseUrl",
  "baseUrl": "https://localhost:8000/app/",
  "env": {
    "_comment": "Default values for these three variables are defined in index.js and can be controlled via environment variables.",
    "baseUrl": "${BASE_URL}",
    "internalUrl": "${INTERNAL_URL}",
    "project": "${PROJECT}"
  },
  "defaultCommandTimeout": 12000,
  "reporter": "junit",
  "reporterOptions": {
    "mochaFile": "build/reports/cypress-${PROJECT}/output-[hash].xml",
    "toConsole": false
  },
  "retries": {
    "runMode": 1,
    "openMode": 1
  },
  "supportFile": "cypress/support/index.ts",
  "video": true,
  "testFiles": [
    "**/common/*.spec.*",
    "**/${PROJECT}/*.${PROJECT}.spec.*"
  ],
  "videoUploadOnPasses": false,
  "experimentalStudio": true
}
mpressmar commented 3 years ago

Just noticed the warning of potentially incompatible Cypress versions, stemming from having overwritten the environment variable CYPRESS_INSTALL_BINARY with a different version. I'm fixing that issue, and will report if it fixes the problem.

hadalin commented 3 years ago

I installed cypress-terminal-report and it started behaving. Quantum mechanics right there.

mpressmar commented 3 years ago

So far this issue didn't appear again. I'm hopeful that the version mismatch caused it :-)