WebKit 18 fails driver-integration-tests-webkit (cookies.cy.js)

Current behavior

If an attempt is made to update playwright-webkit to the latest version in Cypress source code, it fails the Cypress test spec

packages/driver/cypress/e2e/commands/cookies.cy.js

which is part of driver-integration-tests-webkit.

This is reproducible on Ubuntu with versions of playwright-webkit starting with 1.37.0 and extending to the current version (with WebKit ~~17.4~~ 18.0).

Cypress is currently configured to use playwright-webkit@1.24.2 released in July 2022:

https://github.com/cypress-io/cypress/blob/d8884e0a342e92a41fb3e187a807ae7babab3cc7/package.json#L189

Desired behavior

WebKit should pass all tests from driver-integration-tests-webkit when using the latest version of playwright-webkit.

Qualify if this is due to bug(s) in Playwright/WebKit or Cypress
Depending on outcome, report bugs to Playwright and / or fix bugs in Cypress

Test code to reproduce

npm install yarn@latest -g
git clone https://github.com/cypress-io/cypress
cd cypress
yarn add playwright-webkit@1.46.1 -D -W
yarn playwright install-deps webkit # Linux only
yarn workspace @packages/driver cypress:run --browser webkit --spec cypress/e2e/commands/cookies.cy.js

yarn workspace @packages/driver cypress:open --e2e --browser webkit

and select cypress/e2e/commands/cookies.cy.js to run in UI.

Playwright Version

First reported on playwright-webkit@1.45.3 Continues to be reproducible on playwright-webkit@1.47.0

Cypress Version

First reported on 13.13.2 Continues to be reproducible on 13.14.2

Node version

v18.17.1

Operating System

~~Ubuntu 22.04.4 LTS~~ Ubuntu 22.04.5 LTS

Debug Logs

See failures in CircleCI at https://app.circleci.com/pipelines/github/cypress-io/cypress/63152/workflows/e49a5390-7dc4-432e-abce-391b205c080b/jobs/2615104

  1) src/cy/commands/cookies - no stub
       #getCookies
         returns cookies for the specified domain:

      AssertionError: expected 'key2' to equal 'key1'
      + expected - actual

      -'key2'
      +'key1'

      at  ([native code])
      at  (http://www.barbaz.com:3500/__cypress/tests:1717:41)
  getRet@
  tryCatcher@
  @
  thenFn@
  @
  tryCatcher@
  @
  @
  @
  _drainQueueStep@
  _drainQueue@
  @
  @

  2) src/cy/commands/cookies - no stub
       #getCookie
         returns the cookie from the specified domain:

      AssertionError: expected 'foobar.com value' to equal 'www.foobar.com value'
      + expected - actual

      -'foobar.com value'
      +'www.foobar.com value'

      at  ([native code])
      at  (http://www.barbaz.com:3500/__cypress/tests:1870:38)
  getRet@
  tryCatcher@
  @
  thenFn@
  @
  tryCatcher@
  @
  @
  @
  _drainQueueStep@
  _drainQueue@
  @
  @

  3) src/cy/commands/cookies
       #setCookie
         can set cookies with sameSite:

      AssertionError: expected { Object (name, value, ...) } to have property 'sameSite' of 'no_restriction', but got 'lax'
      + expected - actual

      -'lax'
      +'no_restriction'

      at  ([native code])
  applyChainer@
  arrayReduce@
  reduce@
  applyChainers@
  tryCatcher@
  @
  shouldFn@
  @
  tryCatcher@
  @
  @
  @
  _drainQueueStep@
  _drainQueue@
  @
  @

Other

See WebKit (Experimental)
https://playwright.dev/docs/browsers#webkit
See https://github.com/cypress-io/cypress/pull/29955 for failed attempt to update.

Playwright	pass	fail	WebKit	Notes
1.24.2	129	0	16.0	(1)
1.34.3	129	0	16.4	(2)
1.35.0	129	0	16.4	(3)
1.36.2	129	0	17.0	(3)
1.37.0	128	1	17.0	(4)
1.37.1	127	2	17.0	(5)
1.38.1	127	2	17.0	(5)
1.39.0	127	2	17.4	(5)
1.40.1	128	1	17.4	(4)
1.41.2	127	2	17.4	(5)
1.42.1	127	2	17.4	(5)
1.43.1	127	2	17.4	(5)
1.44.1	127	2	17.4	(5)
1.45.0	127	2	17.4	(6)
1.45.1	126	3	17.4	(7)
1.45.3	126	3	17.4	(7)
1.46.0	127	3	18.0	(7)

cypress-io / cypress