cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.72k stars 3.16k forks source link

Safari/webkit tests fail with "page.goto: An SSL error has occurred and a secure connection to the server cannot be made" #25840

Open GrayedFox opened 1 year ago

GrayedFox commented 1 year ago

Current behavior

All of my tests run fine locally using Chrome, Electron, Chromium, and Edge. They mostly work with Firefox (I skip a bunch that rely on iFrames) -- but I cannot get any of them to run on Safari.

The error is this:

page.goto: An SSL error has occurred and a secure connection to the server cannot be made.
=========================== logs ===========================
navigating to "https://artifacts.carted.dev/__/#/specs/runner?file=src/integration/cart-crud.cy.ts", waiting until "load"
============================================================
page.goto: An SSL error has occurred and a secure connection to the server cannot be made.
=========================== logs ===========================
navigating to "https://artifacts.carted.dev/__/#/specs/runner?file=src/integration/cart-crud.cy.ts", waiting until "load"
============================================================
    at b.reset (<embedded>:2760:14600)
    at async b.create (<embedded>:2760:13873)
    at async Object.g (<embedded>:2760:20738)
    at async Object.open (<embedded>:2760:25227)
    at async v.relaunchBrowser (<embedded>:2812:346666)

It's unclear why this happens as the actual URL in question is exposed -- and has a valid SSL cert. Is it possible that Cypress isn't passing the URL correctly to the underlying Playwright instance?

https://artifacts.carted.dev/__/#/specs seems incorrect (expecting to see entire URL plus path).

Desired behavior

Run tests in Safari as expected.

Test code to reproduce

The easiest way to reproduce this is with the following tests. Make sure to pass the correct baseUrl when running the tests.

// passes if baseUrl set to https://docs.cypress.io
describe('super simple test', () => {
  it('navigates to the target URL', () => {
    cy.visit('https://docs.cypress.io/api/commands/visit');
    cy.url().then((string) => expect(string === 'https://docs.cypress.io/api/commands/visit', 'Yay'));
  });
});
// fails even if baseUrl set to https://artifacts.carted.dev
describe('super simple test', () => {
  it('navigates to the target URL', () => {
    cy.visit('https://artifacts.carted.dev/elements/exported/1559/dist/index.html');
    cy.url().then((string) => expect(string === 'https://artifacts.carted.dev/elements/exported/1559/dist/index.html', 'Yay'));
  });
});

That exposed build URL has a valid SSL certificate and I can access it using Safari on my MacBook just fine.

Cypress Version

11.2.0

Node version

18.9.1

Operating System

Mac Os Ventura 13.2 (x86)

Debug Logs

https://pastebin.com/7AdwacJd

Other

Above is the output using DEBUG=cypress:server*

GrayedFox commented 1 year ago

I have a feeling it's due to the injected product cards on the test page -- there is something about either PlayWright or Cypress not liking this.

flotwig commented 1 year ago

It's a strange one, since we do pass ignoreHTTPSErrors to PW: https://github.com/cypress-io/cypress/blob/ed0668e24c2ee6753bbd25ae467ce94ae5857741/packages/server/lib/browsers/webkit-automation.ts#L99

Furthermore, I am able to launch your site just fine in regular pw-webkit:

pw.webkit.launch({ ignoreHTTPSErrors: true, headless: false }).then(b => b.newPage()).then(p => p.goto('https://artifacts.carted.dev/elements/exported/1559/dist/index.html'))

image

So it seems to be an issue specific to Cypress and Playwright working together. I'm not sure what it could be. I'll get this prioritized for further investigation and a fix.

avostretsov commented 1 year ago

I am seeing similar issue in my project. Screenshot 2023-07-17 at 3 16 24 PM

Bohdan-Hevlich commented 1 year ago

Have same issue in my project. Cypress v 12.10.0, playwright-webkit: "1.34". page.goto: An SSL error has occurred and a secure connection to the server cannot be made. =========================== logs =========================== navigating to url/__/#/specs/runner?file=cypress/e2e/attendance/test.cy.js", waiting until "load"

MarkoAssertQA commented 3 months ago

We just came across the same issue. The test app also has the ".dev" domain

We were developing on OSX Sonoma which still reroutes .dev domains to localhost or whatever. We just executed the same test repository on Windows and it did not fire this error.

p.s. On Sonoma with Cypress and WebKit try to do a cy.visit("https://playwright.dev/") or flutter.dev