Cypress hides the error pages of visited websites

[sharmilajesupaul]

Current behavior

If you run a test on a website that has a error pages with rendered content, Cypress will hide the page and show a blank page with the error code on it.

So on airbnb.com, instead of seeing this on 4XX errors:

I see this in Cypress:

Desired behavior

We recently began showing helpful debugging information on our error pages in our dev environments. However, when we come across these errors in Cypress tests, the debugging info on the page is hidden by Cypress. We see a blank page with the error code instead which is not helpful.

I do not want the test behavior to change, the test should still fail on an incorrect status code. However, I'd like Cypress to stop hiding the page being rendered by the website because it could contain meaningful information.

Test code to reproduce

it('visits a broken page', () => {
  cy.visit('https://www.airbnb.com/thisdoesntexist');
  expect(true).to.equal(true);
});

Cypress Version

12.17.3

Node version

16.20.0

Operating System

macOS Ventura 13.5 (22G74)

Debug Logs

No response

Other

No response

[chrisbreiding]

We'll take this under consideration as a feature request.

Currently, you could use the failOnStatusCode option to keep the visit from failing so that Cypress displays the page. Your test will likely still fail on subsequent commands/assertions.

it('visits a broken page', () => {
  cy.visit('https://www.airbnb.com/thisdoesntexist', { failOnStatusCode: false });
  cy.contains(`We can’t seem to find the page you’re looking for`).should('not.exist') // or an assertion for something that should be on the page if it didn't 404
});

[sharmilajesupaul]

Currently, you could use the failOnStatusCode option to keep the visit from failing so that Cypress displays the page.

@chrisbreiding We have tests that are relying on the status code failure behavior. We also want to do this in a global way for all our projects so all our tests can see debugging info on error pages when they run on our dev environments.

Do you know if this is a case where I can overwrite cy.visit via Cypress.Commands.overwrite('visit',...) to get it to behave this way?

[chrisbreiding]

Do you know if this is a case where I can overwrite cy.visit via Cypress.Commands.overwrite('visit',...) to get it to behave this way?

Yes, you can overwrite cy.visit so that it always includes the option when it calls through to the actual cy.visit.

[sharmilajesupaul]

Turns out that it's not easy at all to do overwrite the command in such a way that you can:

1. You can show the page that's being hidden by Cypress
2. Still fail the cy.visit command 😕

The way it works, is the command will either return an error message string or the window object but does not expose any other info about the result of the visit.

An alternative would be to send down the page response with window to end users, that way we can at least pull meaningful data out of the response object. is that something you would consider @chrisbreiding?

[sharmilajesupaul]

ok this is my prototype of how I want it to behave, using fetch to try and make up for the lack of response data:

Cypress.Commands.overwrite(
  'visit',
  (originalFn, url, options) => {
    return fetch(typeof url === 'string' ? url : url.url, {
      method: 'HEAD', // Use the HEAD method to avoid downloading the body
    }).then((response) => {
      // Check if the HTTP status code is in the range of success statuses
      if (response.ok) {
        // If the status code is 200-299, it's okay to navigate
        return originalFn({ ...options, url });
      }
      // Make the visit anyway, but don't fail the test on it yet
      originalFn({ ...options, url, failOnStatusCode: false });
      // Handle non-successful HTTP status codes as needed
      throw new Error(`
      The URL ${url} is not reachable.
      HTTP status code: ${response.status}
    `);
    });
  },
);

There might be issues with fetch because it's a promise, but I'm not sure how else to go about this 🤔

[sharmilajesupaul]

I guess this completely messes up the stack trace too, like the errors will now always point at the throw in the override