Issue with Github Actions not closing/finishing when testing with playwright-webkit (Safari)

tnelms1 commented 1 year ago

Current behavior

When testing with playwright-webkit@1.35.0 after the Cypress test completes the instance just pauses until Github times it out

Desired behavior

In previous versions (1.34.0) the test completes as it does for all my other GitHub Action Cypress test (non-safari)

(non-sa

Test code to reproduce

https://github.com/SmartBear/visualtest-cypress visualtest-cypress/test/webkit-test the test in that e2e is the one that fails

it is ran from a YAML file: https://github.com/SmartBear/visualtest-cypress/blob/main/.github/workflows/webkit-browser_test.yml I have changed the yaml file: https://github.com/SmartBear/visualtest-cypress/blob/dce910895b546dd7d6e90735733a5c33a800cbb2/.github/workflows/webkit-browser_test.yml#L25C14-L25C50 from run: npm install playwright-webkit@latest to run: npm install playwright-webkit@1.34.0

Cypress Version

12.16.0

Node version

v16.18.0

Operating System

macOS 13.4 && GitHub Ubuntu-latest

Debug Logs

bit' } +5ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] 264 - core 157 r2935 545de2f - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=6 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=30 rc=crf mbtree=1 crf=32.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00' } +0ms
  cypress:server:video compression stderr log { message: "Output #0, mp4, to '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/videos/spec.cy.js-compressed.mp4':" } +0ms
  cypress:server:video compression stderr log { message: '  Metadata:' } +0ms
  cypress:server:video compression stderr log { message: '    encoder         : Lavf58.24.101' } +0ms
  cypress:server:video compression stderr log { message: '    Chapter #0:0: start 0.000000, end 5.917000' } +0ms
  cypress:server:video compression stderr log { message: '    Metadata:' } +0ms
  cypress:server:video compression stderr log { message: '      title           : tests a few captures on safari passes' } +0ms
  cypress:server:video compression stderr log { message: '    Stream #0:0(eng): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 24.33 fps, 18688 tbn, 24.33 tbc (default)' } +0ms
  cypress:server:video compression stderr log { message: '    Metadata:' } +0ms
  cypress:server:video compression stderr log { message: '      encoder         : Lavc58.42.102 libx264' } +1ms
  cypress:server:video compression stderr log { message: '    Side data:' } +0ms
  cypress:server:video compression stderr log { message: '      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1' } +0ms
  cypress:server:video compression stderr log { message: 'frame=  129 fps=0.0 q=34.0 size=       0kB time=00:00:08.21 bitrate=   0.1kbits/s speed=16.1x    ' } +484ms
  cypress:server:video compression progress: { frames: 129, currentFps: 0, currentKbps: 0.1, targetSize: 0, timemark: '00:00:08.21' } +0ms
  cypress:server:video compression stderr log { message: 'frame=  303 fps=296 q=35.0 size=     512kB time=00:00:17.17 bitrate= 244.2kbits/s speed=16.8x    ' } +514ms
  cypress:server:video compression progress: { frames: 303, currentFps: 296, currentKbps: 244.2, targetSize: 512, timemark: '00:00:17.17' } +0ms
  cypress:server:video compression stderr log { message: 'frame=  457 fps=293 q=-1.0 Lsize=    1204kB time=00:00:27.86 bitrate= 353.9kbits/s speed=17.9x    ' } +533ms
  cypress:server:video compression progress: { frames: 457, currentFps: 293, currentKbps: 353.9, targetSize: 1204, timemark: '00:00:27.86' } +0ms
  cypress:server:video compression stderr log { message: 'video:1196kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.625363%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] frame I:18    Avg QP:24.26  size: 15080' } +3ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] frame P:160   Avg QP:29.15  size:  4929' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] frame B:279   Avg QP:31.70  size:   589' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] consecutive B-frames: 12.3% 15.3% 11.2% 61.3%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] mb I  I16..4: 37.2% 50.4% 12.5%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] mb P  I16..4:  6.1% 11.0%  1.6%  P16..4:  7.4%  1.2%  0.7%  0.0%  0.0%    skip:71.9%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] mb B  I16..4:  1.5%  0.2%  0.0%  B16..8:  4.2%  0.3%  0.0%  direct: 2.4%  skip:91.4%  L0:48.0% L1:49.8% BI: 2.2%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] 8x8 transform intra:51.6% inter:24.3%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] coded y,uvDC,uvAC intra: 12.1% 7.8% 1.9% inter: 1.4% 2.6% 0.2%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] i16 v,h,dc,p: 49% 40% 10%  1%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 48% 17% 32%  0%  0%  0%  0%  0%  1%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 31% 19%  2%  3%  3%  3%  3%  4%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] i8c dc,h,v,p: 79% 15%  5%  0%' } +1ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] Weighted P-Frames: Y:2.5% UV:0.0%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] ref P L0: 72.9% 27.1%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] ref B L0: 85.3% 14.7%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] ref B L1: 95.3%  4.7%' } +0ms
  cypress:server:video compression stderr log { message: '[libx264 @ 0x7fb9bd00d600] kb/s:345.42' } +0ms
  cypress:server:video compression stderr log { message: '' } +38ms
  cypress:server:video compression ended +0ms
  -  Finished compressing: 1 second                                                  

  -  Video output: /Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/videos/spec.cy.js.mp4

  cypress:server:run spec results { stats: { suites: 1, tests: 1, passes: 1, pending: 0, skipped: 0, failures: 0, wallClockStartedAt: 2023-06-26T20:23:11.829Z, wallClockEndedAt: 2023-06-26T20:23:35.799Z, wallClockDuration: 23970 }, reporter: 'spec', reporterStats: { suites: 1, tests: 1, passes: 1, pending: 0, failures: 0, start: 2023-06-26T20:23:11.843Z, end: 2023-06-26T20:23:35.832Z, duration: 23989 }, hooks: [], tests: [ { testId: 'r3', title: [Array], state: 'passed', body: '() => {\n' + "    cy.visit('https://example.cypress.io');\n" + "    cy.sbvtCapture('webkit-fullpage');\n" + "    cy.sbvtCapture('webkit-viewport', {\n" + "      capture: 'viewport'\n" + '    });\n' + "    cy.get('.banner').sbvtCapture('webkit-element');\n" + '    cy.sbvtPrintReport();\n' + '  }', displayError: null, attempts: [Array] } ], error: null, video: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/videos/spec.cy.js.mp4', screenshots: [ { screenshotId: '18jjv', name: 'tmp/webkit-fullpage/0', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:15.923Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/0.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: '7fsvx', name: 'tmp/webkit-fullpage/1', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:16.479Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/1.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: 'k1qyj', name: 'tmp/webkit-fullpage/2', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:16.858Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/2.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: 't61p0', name: 'tmp/webkit-fullpage/3', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:17.228Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/3.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: 'fmdar', name: 'tmp/webkit-fullpage/4', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:17.588Z'
, path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/4.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: '6o9ad', name: 'tmp/webkit-fullpage/5', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:18.011Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/5.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: 'eei7u', name: 'tmp/webkit-fullpage/6', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:18.385Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/6.png', height: 660, width: 1000, pathname: undefined }, { sc====================================================================================================
reenshotId: 'f1dhh', name: 'tmp/webkit-fullpage/7', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:18.815Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/tmp/webkit-fullpage/7.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: '0tu6d', name: 'webkit-viewport', testId: 'r3', testAttemptIndex: 0, takenAt: '2023-06-26T20:23:25.047Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/webkit-viewport.png', height: 660, width: 1000, pathname: undefined }, { screenshotId: 'yi4sb', name: 'webkit-element', testId: 'r3', testAttemptIndex: 0, takenAt: 
                                                    [32m'2023-06-26T20:23:28.999Z', path: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots/spec.cy.js/webkit-element.png', height: 189, width: 1000, pathname: undefined } ], spec: { fileExtension: '.js', baseName: 'spec.cy.js', fileName: 'spec', specFileExtension: '.cy.js', relativeToCommonRoot: 'spec.cy.js', specType: 'integration', name: 'cypress/e2e/spec.cy.js', relative: 'cypress/e2e/spec.cy.js', absolute: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/e2e/spec.cy.js' }, shouldUploadVideo: true } +2s
  (Run Finished)
  cypress:server:run final results of all runs: { status: 'finished', startedTestsAt: 2023-06-26T20:23:11.829Z, endedTestsAt: 2023-06-26T20:23:35.799Z, totalDuration: 23970, totalSuites: 1, totalTests: 1, totalPassed: 1, totalPending: 0, totalFailed: 0, totalSkipped: 0, runs: [ { stats: [Object], reporter: 'spec', reporterStats: [Object], hooks: [], tests: [Array], error: null, video: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/videos/spec.cy.js.mp4', screenshots: [Array], spec: [Object], shouldUploadVideo: true } ], browserPath: '/Users/trevor.nelms/Library/Caches/ms-playwright/webkit-1860/pw_run.sh', browserName: 'webkit', browserVersion: '16.4', osName: 'darwin', osVersion: '22.5.0', cypressVersion: '12.14.0', runUrl: undefined, config: { experimentalWebKitSupport: true, setupNodeEvents: '[Function setupNodeEvents]', projectRoot: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test', projectName: 'webkit-
test', repoRoot: '/Users/trevor.nelms/Desktop/visualtest-cypress', rawJson: { experimentalWebKitSupport: true, e2e: [Object], setupNodeEvents: '[Function setupNodeEvents]', envFile: {}, projectRoot: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test', projectName: 'webkit-test', repoRoot: '/Users/trevor.nelms/Desktop/visualtest-cypress' }, configFile: 'cypress.config.js', morgan: false, isTextTerminal: true, socketId: '9g5nbelmsg', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: null, blockHosts: null, chromeWebSecurity: true, clientCertificates: [], defaultCommandTimeout: 4000, downloadsFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/downloads', env: {}, execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRu
nEvents: false, experimentalRunAllSpecs: false, experimentalMemoryManagement: false, experimentalModifyObstructiveThirdPartyCode: false, experimentalSkipDomainInjection: null, experimentalOriginDependencies: false, experimentalSourceRewriting: false, experimentalSingleTabRunMode: false, experimentalStudio: false, fileServerFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test', fixturesFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/fixtures', excludeSpecPattern: '*.hot-update.js', includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, nodeVersion: undefined, numTestsKeptInMemory: 0, platform: 'darwin', pageLoadTimeout: 60000, port: 64820, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/usr/local/bin/node', resolvedNodeVersion: '16.18.0', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, screenshotsFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/screenshots', slowTestThreshold: 10000, scrollBehavior: 'top', supportFile: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/support/e2e.js', supportFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/support', taskTimeout: 60000, testIsolation: true, trashAssetsBeforeRuns: true, userAgent: null, video: true, videoCompression: 32, videosFolder: '/Users/trevor.nelms/Desktop/visualtest-cypress/test/webkit-test/cypress/videos', videoUploadOnPasses: true, viewportHeight: 660       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  spec.cy.js                               00:23        1        1        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:23        1        1        -        -        -  
, viewportWidth: 1000, waitForAnimations: true, watchForFileChanges: false, specPattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}', additionalIgnorePattern: [], autoOpen: false, browsers: [ [Object], [Object], [Object], [Object], [Object] ], clientRoute: '/__/', cypressBinaryRoot: '/Users/trevor.nelms/Library/Caches/Cypress/12.14.0/Cypress.app/Contents/Resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '12.14.0', cypressEnv: 'production', resolved: { animationDistanceThreshold: [Object], arch: [Object], baseUrl: [Object], blockHosts: 
[Object], chromeWebSecurity: [Object], clientCertificates: [Object], defaultCommandTimeout: [Object], downloadsFolder: [Object], env: {}, execTimeout: [Object], experimentalFetchPolyfill: [Object], experimentalInteractiveRunEvents: [Object], experimentalRunAllSpecs: [Object], experimentalMemoryManagement: [Object], experimentalModifyObstructiveThirdPartyCode: [Object], experimentalSkipDomainInjection: [Object], experimentalOriginDependencies: [Object], experimentalSourceRewriting: [Object], experimentalSingleTabRunMode: [Object], experimentalStudio: [Object], experimentalWebKitSupport: [Object], fileServerFolder: [Object], fixturesFolder: [Object], excludeSpecPattern: [Object], includeShadowDom: [Object], keystrokeDelay: [Object], modifyObstructiveCode: [Object], nodeVersion: [Object], numTestsKeptInMemory: [Object], platform: [Object], pageLoadTimeout: [Object], port: [Object], projectId: [Object], redirectionLimit: [Object], reporter: [Object], reporterOptions: [Object], requestTimeout: [Object], resolvedNodePath: [Object], resolvedNodeVersion: [Object], responseTimeout: [Object], retries: [Object], screenshotOnRunFailure: [Object], screenshotsFolder: [Object], slowTestThreshold: [Object], scrollBehavior: [Object], supportFile: [Object], supportFolder: [Object], taskTimeout: [Object], testIsolation: [Object], trashAssetsBeforeRuns: [Object], userAgent: [Object], video: [Object], videoCompression: [Object], videosFolder: [Object], videoUploadOnPasses: [Object], viewportHeight: [Object], viewportWidth: [Object], waitForAnimations: [Object], watchForFileChanges: [Object], specPattern: [Object], browsers: [Object], hosts: [Object], isInteractive: [Object] }, testingType: 'e2e', remote: { auth: undefined, origin: 'http://localhost:64820', strategy: 'file', fileServer: 'http://localhost:64821', domainName: 'localhost', props: null }, browser: null, specs: [], proxyUrl: 'http://localhost:64820', browserUrl: 'http://localhost:64820/__/', reporterUrl: 'http://localhost:64820/__cypress/reporter', proxyServer: 'http://localhost:64820', state: {} } } +0ms
  cypress:lifecycle:EventRegistrar plugin event registered? { eventName: 'after:run', isRegistered: false } +2s
  cypress:server:cypress about to exit with code 0 +36s
  cypress:server:browsers browsers.kill called with no active instance +28s
  cypress:proxy:http:util:prerequests metrics: { browserPreRequestsReceived: 45, proxyRequestsReceived: 9, immediatelyMatchedRequests: 9, unmatchedRequests: 0, unmatchedPreRequests: 34 } +0ms

Other

It pulls up this at the end of a npx cypress run -b webkit test

mike-plummer commented 1 year ago

Hi @tnelms1 , are you seeing this problem with Cypress even when running outside of the Github Action? If it's only occurring within the action then it may be an issue with that package, in which case I would recommend reporting this issue there

tnelms1 commented 1 year ago

Hi @mike-plummer & @lmiller1990 , sorry for the delay was on holiday. When not run on the GitHub Action after the tests finish an odd pop-up occurs where it asks to add project (it is pictured above at the very bottom of my ticket request)

Also, this issue has not been resolved since 12.17.1, I tested today

lmiller1990 commented 1 year ago

I reproduced, nothing to do with GHA... general bug. We have seen this before, though, let me dig up a commit that was supposedly meant to fix it. The hanging is Cypress opening. It was introduced somewhere in PW 1.34 and 1.35.

https://github.com/cypress-io/cypress/assets/19196536/99f1e83c-8b8a-446e-8ca4-7a5753b6364b

lmiller1990 commented 1 year ago

X 1.35.0-alpha-jun-3-2023
X 1.35.0-alpha-jun-2-2023
X 1.35.0-alpha-jun-1-2023
X 1.35.0-alpha-1685749070000
O 1.35.0-alpha-1685109821000
O 1.35.0-alpha-1685046878000
O 1.34.3

Looks like we broke between 1.35.0-alpha-1685109821000 -> 1.35.0-alpha-1685749070000.

I do not see a release with that tag, though. Best bet might be find out the exact publication date of both via npm, and see what commits happened around that time.

lmiller1990 commented 1 year ago

I found it the fix for that original bug. We hit this same issue during initial webkit development, here's the line that fixed it then: https://github.com/cypress-io/cypress/pull/23662/files#r963165698

This should help when someone takes a look at this. Would you be interested in taking a look @tnelms1?

tnelms1 commented 1 year ago

Glad the issue was identified. @lmiller1990 I don't believe I have proper time, with day-job + life

MikeMcC399 commented 1 year ago

I had a look at adding webkit to the set of browser examples on cypress-io/github-action to establish a baseline, however the webkit implementation seems to be more on the experimental end of the scale at the moment and not mature enough to set up a solid example.

Firing up the Cypress app when npx cypress run has finished is quite confusing. I think this needs to be resolved first.
Also running on Ubuntu with the Cypress app npx cypress open and not getting a list of test specs is something which needs to be sorted out (https://github.com/cypress-io/cypress/issues/24767). That should work out-of-the-box.

lmiller1990 commented 1 year ago

We are looking at getting this fixed internally. Please wait a bit while we prioritize the issue.

lmiller1990 commented 1 year ago

For now I'm going to recommend using the playwright-webkit verison that is known to work, 1.34. cypress-io/cypress-documentation#5375

mkascel commented 1 year ago

I believe I am also running into this issue on CircleCI so wanted to add my debugging info in case it is useful or if the solution in the works is targeting GHA only. My Chrome tests exit after success or fail as expected but the exit code doesn't seem to be communicated back to Circle. Debug info after the job failure:

  cypress:server:cypress about to exit with code 2 +2m
  cypress:server:browsers browsers.kill called with no active instance +2m
  cypress:proxy:http:util:prerequests metrics: { browserPreRequestsReceived: 418, proxyRequestsReceived: 388, immediatelyMatchedRequests: 322, unmatchedRequests: 66, unmatchedPreRequests: 96 } +0ms

Build was canceled

The build-cancelation happens as a result of CI timeout or manual cancel.

MikeMcC399 commented 1 year ago

@mkascel

... or if the solution in the works is targeting GHA only.

I haven't seen anything so far to indicate that a solution would only work on GitHub Actions. In fact in https://github.com/cypress-io/cypress/issues/27141#issuecomment-1630179586 it says:

I reproduced, nothing to do with GHA... general bug.

cypress-app-bot commented 7 months ago

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

MikeMcC399 commented 6 months ago

Testing on

Ubuntu 22.04.4 LTS
Node.js v20.11.1 LTS
Cypress 13.6.6

npx cypress run --e2e --browser webkit

After executing the above command, Cypress incorrectly launches in interactive (open) project mode:

but not with the recommended old version playwright-webkit@1.34.0 v16 (16.4)
continues to be reproducible with playwright-webkit@1.35.1 with WebKit v16 (16.4)
and is no longer reproducible with playwright-webkit@1.41.2 (latest) with WebKit v17 (17.4)

So, possibly this issue should be closed?

Guides > Launching Browsers > WebKit (Experimental) says:

Install the playwright-webkit NPM package in your repo to acquire WebKit itself: npm install --save-dev playwright-webkit@1.34.

MikeMcC399 commented 1 month ago

@tnelms1

Is this issue still reproducible for you if you use latest versions?

cypress@13.13.1
playwright-webkit@1.45.3

cypress-io / cypress