search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.69k stars 3.16k forks source link

Interceptor doesn't intercept API calls which sent from previous test #26681

Open ChenKuanSun opened 1 year ago

ChenKuanSun commented 1 year ago

Current behavior

In our team, we want to maintain the test context and continue with the next test after one test has ended while Test isolation is disabled.

However, if we have some background services like the User data service, they might make some API calls between each test. If we use a setup to mock our API calls, and if the service issues an API call after the previous 'it' scope test ends but before the next mock API function is set up, then this API call will not be intercepted.

Desired behavior

If Test isolation is disabled, the API requests that were originally set up with an interceptor should still be intercepted as expected.

Test code to reproduce

Sample repo

Sample code

/// <reference types="cypress" />

/**
 * Before each test, we are mocking all the API call and change the response.
 * But if there is a api call in the background like something service,
 * the intercept will fail due to Cypress's netStubbingState reset.
 *
 * Note: we are disabling the testIsolation to keep context between tests.
 */
describe("Situations where intercept fails in our app", () => {
  const setup = (delayTime = 0) => {
    // Initial Cypress Commands takes sometime.
    cy.wait(delayTime);
    // Intercept the API call
    cy.intercept(
      {
        pathname: "/",
        method: "GET",
      },
      (req) => {
        req.reply({
          statusCode: 200,
          body: JSON.stringify("intercepted api"),
        });
      }
    ).as("getData");
  };

  it("visits the app", () => {
    // Visit the app
    cy.visit("https://main.d260hbq9idig9k.amplifyapp.com/");

    // Check if the app is loaded
    cy.get("#welcome h1").should("contain", "Welcome cypress-sample-page");
  });

  it("intercepts and changes the response", () => {
    // Setup the test
    setup();

    // Click the button to call the API
    cy.get("#call-api").click();

    // Wait for the API call to complete
    cy.wait(["@getData"]);

    // Check if the response is intercepted
    cy.get("#response pre").should("contain", "intercepted api");

    // Call the API again but we don't wait for the response
    cy.get("#call-api-delay-1-second").click();

    // Then end the test immediately
  });

  it("intercept failed", () => {
    // Setup the test
    setup(
      1500 // Wait for 1.5 seconds to make sure the previous API call is emited
    );

    // Click the button to call the API check if the response is intercepted
    cy.get("#call-api").click();

    // Wait for the API call to complete
    cy.wait(["@getData"]);

    // Set wait time to 3 seconds make sure the all API call is completed
    cy.wait(3000);

    cy.get("#response pre").should("not.contain", "Hello from server!");
  });
});

Cypress Version

12.8.1

Node version

v18.15.0

Operating System

macOS 13.3.1

Debug Logs

cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest proxying request { req: { method: 'GET', proxiedUrl: 'https://secretUrl/xxx', headers: { host: 'secretUrl', connection: 'keep-alive', accept: 'application/json', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/113.0.5672.63 Safari/537.36', 'sec-ch-ua': '"HeadlessChrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'x-cypress-is-xhr-or-fetch': 'xhr', 'sec-fetch-site': 'cross-site', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', referer: 'http://localhost:4200/', 'accept-encoding': 'gzip, deflate, br' } } } +540ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest found x-cypress-is-xhr-or-fetch header. Deleting x-cypress-is-xhr-or-fetch header. +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest waiting for prerequest +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest cy.intercept: intercepting request +5ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse received response { req: { method: 'GET', proxiedUrl: 'https://secretUrl/xxx', headers: { host: 'secretUrl', connection: 'keep-alive', accept: 'application/json', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/113.0.5672.63 Safari/537.36', 'sec-ch-ua': '"HeadlessChrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'sec-fetch-site': 'cross-site', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', referer: 'http://localhost:4200/', 'accept-encoding': 'gzip, deflate, br', 'content-length': '0' } }, incomingRes: { headers: { 'content-type': 'application/json', 'access-control-allow-origin': '*', 'access-control-allow-credentials': 'true' }, statusCode: 200 } } +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse ensuring resStream is plaintext +1ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse determine injection +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse - no injection (not html) +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse injection levels: { isInitial: false, wantsInjection: false, wantsSecurityRemoved: false } +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse cy.intercept: request/response finished, cleaning up +1ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest proxying request { req: { method: 'GET', proxiedUrl: 'https://secretUrl/xxx', headers: { host: 'secretUrl', connection: 'keep-alive', accept: 'application/json', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/113.0.5672.63 Safari/537.36', 'sec-ch-ua': '"HeadlessChrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'x-cypress-is-xhr-or-fetch': 'xhr', 'sec-fetch-site': 'cross-site', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', referer: 'http://localhost:4200/', 'accept-encoding': 'gzip, deflate, br' } } } +542ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest found x-cypress-is-xhr-or-fetch header. Deleting x-cypress-is-xhr-or-fetch header. +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingRequest waiting for prerequest +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse received response { req: { method: 'GET', proxiedUrl: 'https://secretUrl/xxx', headers: { host: 'secretUrl', connection: 'keep-alive', accept: 'application/json', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/113.0.5672.63 Safari/537.36', 'sec-ch-ua': '"HeadlessChrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'sec-fetch-site': 'cross-site', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', referer: 'http://localhost:4200/', 'accept-encoding': 'gzip' } }, incomingRes: { headers: { 'content-type': 'application/json', 'content-length': '0', connection: 'keep-alive', date: 'Fri, 05 May 2023 19:04:10 GMT', 'access-control-allow-origin': '*', ... , statusCode: 401 } } +505ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse determine injection +1ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse - no injection (not html) +0ms
  cypress-verbose:proxy:http GET https://secretUrl/... IncomingResponse injection levels: { isInitial: false, wantsInjection: false, wantsSecurityRemoved: false } +0ms

Other

No response

nagash77 commented 1 year ago

Hi @ChenKuanSun can you please create a reproducible example suing Cypress Test Tiny and update this ticket with that example?

ChenKuanSun commented 1 year ago

Hi @ChenKuanSun can you please create a reproducible example suing Cypress Test Tiny and update this ticket with that example?

Hello @nagash77 I have created repo from template.

https://github.com/ChenKuanSun/cypress-test-tiny-issue-sample

Note: we are disabling the testIsolation to keep context between tests.

Before each test, we are mocking all the API call and change the response in setup function. But if there is some api calls in the background like something service, the intercept will fail due to Cypress's netStubbingState reset.

chrisbreiding commented 1 year ago

Thanks for providing a reproducible example, @ChenKuanSun. I have replicated your problem in the latest version of Cypress and will forward this ticket to the appropriate team. They will evaluate the priority of this ticket and consider their capacity to pick it up. Please note that this does not guarantee that this issue will be resolved. The ticket will indicate status changes during evaluation, so we ask that you please refrain from asking for updates. Thanks!

vaunus commented 1 year ago

I think we are having a flavour of this problem as well which has cost significant time and frustration debugging before finding this Github issue. It seems like it is the same problem as we're defining an intercept with the same url between tests and waiting on it in each test:

const login = ({ email, password }) => {
  cy.intercept('POST', `${apiUrl}/login`).as('postLogin')
  cy.visit(loginUrl)

  cy.get("[name='login']").type(email)
  cy.get("[type='password']").type(password)
  cy.get('button').contains('Log in').click()
  cy.wait(`@postLogin`, { timeout: 30000 })
    .then((result) => {
      // result is undefined when the test fails
      cy.task('log', JSON.stringify(result))
      cy.wrap(result)
    })
    .its('response.statusCode')
    .should('eq', 200)
}

describe('auth', () => {
  it('should be sent to xxx', () => {
    login({ email, password })
    cy.url().should('eq', `xxx`)
  })

  it('should be sent to yyy', () => {
    login({ email, password })
    cy.url().should('eq', `yyy`)
  })

  it('should be sent to zzz', () => {
    login({ email, password })
    cy.url().should('eq', 'zzz')
  })
})

It consistently works in the first xxx test but then randomly fails in the 2nd or 3rd tests with the cy.wait returning undefined immediately - i.e. it does not wait 30000ms.

Do you know what version of Cypress this was introduced in @ChenKuanSun as I'd like to consider rolling back until this is fixed? If not I will try to determine this for myself.