cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.51k stars 3.2k forks source link

Cypress.automation('remote:debugger:protocol') command started failing in Cypress 7.3.0 #17723

Open c32hedge opened 3 years ago

c32hedge commented 3 years ago

Current behavior

Starting in 7.3.0, the test code below (mostly copied from https://www.cypress.io/blog/2020/11/12/testing-application-in-offline-network-mode/) throws this error:

CypressError: `cy.then()` timed out after waiting `4000ms`.

Your callback function returned a promise that never resolved.

The callback function was:

() => Cypress.automation('remote:debugger:protocol', {
      command: 'Network.emulateNetworkConditions',
      params: {
        offline: true,
        latency: -1,
        downloadThroughput: -1,
        uploadThroughput: -1
      }
    })
    at https://www.cypress.io/__cypress/runner/cypress_runner.js:136167:24
From previous event:
    at Context.thenFn (https://www.cypress.io/__cypress/runner/cypress_runner.js:136166:13)
    at Context.then (https://www.cypress.io/__cypress/runner/cypress_runner.js:136597:21)
    at Context.<anonymous> (https://www.cypress.io/__cypress/runner/cypress_runner.js:151455:21)
From Your Spec Code:
    at Context.eval (https://www.cypress.io/__cypress/tests?p=cypress/integration/spec.js:133:9)

Desired behavior

The test should run without error.

Test code to reproduce

Set the baseUrl configuration key (e.g. "https://www.cypress.io") and run the following spec. I can reproduce without setting the baseUrl but it's a little more complicated and I think this is better as a MWE.

describe('page', () => {
  it('tries to go offline', () => {
    cy.log('**go offline**')
    .then(() =>
      Cypress.automation('remote:debugger:protocol', { command: 'Network.enable' })
    )
      .then(() => 
        Cypress.automation('remote:debugger:protocol', {
          command: 'Network.emulateNetworkConditions',
          params: {
            offline: true,
            latency: -1,
            downloadThroughput: -1,
            uploadThroughput: -1
          }
        })
      );
    cy.visit('/');
  });
});

Cypress Version

7.3.0

Other

I encountered this when attempting to update to the latest version of Cypress, but traced it back to 7.3.0. Version 7.2.0 runs without error.

It looks like at least a couple other people have encountered this based on the comments on that blog post. I found that modifying the callback in the second then to not return anything did allow my tests to get further, only to hang indefinitely at random places in my test file so, as I suspected, that isn't a real solution.

InfiniteTape commented 3 years ago

+1 to this. I'm seeing the same problem.

szhong-becls commented 3 years ago

The same regression issue in 8.0.0

c32hedge commented 3 years ago

@jennifer-shehane did this get lost? It's been two weeks with no response 😦

c32hedge commented 3 years ago

@jennifer-shehane Gleb pointed me to this Cypress recipe on the Cypress Gitter: https://github.com/cypress-io/cypress-example-recipes/blob/master/examples/server-communication__offline/cypress/integration/offline-spec.js

Which is almost identical to my code except for the use of braces and explicit return statements for the arrow function bodies. Yet, I was able to use the MWE above in a cypress-test-tiny fork and trace the failure back to 7.3.0 as described.

I realize this is a bit of an obscure feature, but you have blogs and recipes that reference it, and multiple people, both here and in the comments on the blog post, are struggling getting code referenced in those blogs/recipes to work in newer versions of Cypress. Could you at least confirm whether you can reproduce the error I'm seeing?

szhong-becls commented 3 years ago

According to my investigation, this issue may be injected by this commit in socketio/engine.io-client repo. It supports a listen to the "offline" event feature in version 5.0.0 (2021-03-10).

The problemic code is as follows:

+ if (this.hostname !== "localhost") {
+   this.offlineEventListener = () => {
+     this.onClose("transport close");
+   };
+   addEventListener("offline", this.offlineEventListener, false);
+ }

The socket is closed after sending CDP command: { "command": "Network.emulateNetworkConditions", "parameters": { "offline": true } }. Then the command response is never received and Cypress.automation('remote:debugger:protocol', { ... }) is timeout.

raultm commented 3 years ago

any idea or workaround? Can i get an older version of cypress to make it work?

It only fails when is not localhost, in fact i can go offline but i cant recover from this status to the online status

https://github.com/cypress-io/cypress/issues/17723#issuecomment-906152925

svenjungnickel commented 3 years ago

A quick workaround would be to use a version before cypress 7.0.0. As I stated here https://github.com/cypress-io/cypress-example-recipes/issues/721#issuecomment-920609040 I got it successfully running in version 6.9.1.

szhong-becls commented 3 years ago

Another "work around". With the fact that Network.emulateNetworkConditions and offline: true did set the Chrome network to offline, but Cypress cannot receive the command execution result. So we can tell Cypress to ignore this particular error, and continue to run our test specs.

cy.on('fail', e => {
  if (
    e.name === 'CypressError'
    && e.message.includes('timed out')
    && e.message.includes('remote:debugger:protocol')
    && e.message.includes('Network.emulateNetworkConditions')
    && e.message.includes('offline: true')
  ) {
    // do not fail the test
    return false;
  }
});

cy.log('**go offline**')
  .then(() =>
    Cypress.automation('remote:debugger:protocol', { command: 'Network.enable' })
  )
  .then(() =>
    Cypress.automation('remote:debugger:protocol', {
      command: 'Network.emulateNetworkConditions',
      params: {
        offline: true,
        latency: -1,
        downloadThroughput: -1,
        uploadThroughput: -1
      }
    })
  );
svenjungnickel commented 3 years ago

@szhong-becls your workaround did not work for me, unfortunately.

matthew-nash-cko commented 3 years ago

Any update on this? My tests are also failing where this functionality is required

myasul commented 3 years ago

I am also experiencing this issue on our project when I upgraded to v8.6.0. Is there a provided solution or an ETA for a fix? We cannot upgrade till this is fixed. 😭

lorenzofidalgo commented 3 years ago

I am also suffering with this issue. I have confirmed Cypress 6.9.1 is still working properly. I have tested Cypress 6.9.1 against Chrome 96 and Electron 87 (working properly in both cases). Cypress 8.7.0 fails, Cypress 9.0.0 fails. So, or we go back to 6.9.1, or...

maayanzrihann commented 3 years ago

Same for me :( Cannot upgrade cypress version because of this issue

Gu7z commented 3 years ago

Same! :eyes:

ughunoup commented 2 years ago

Same, still not working in 9.1.1

Do anyone know alternative solution to simulate browser offline mode?

weilu commented 2 years ago

Same here. Stuck with 6.9.1 because of this issue

distante commented 2 years ago

Another "work around". With the fact that Network.emulateNetworkConditions and offline: true did set the Chrome network to offline, but Cypress cannot receive the command execution result. So we can tell Cypress to ignore this particular error, and continue to run our test specs.

cy.on('fail', e => {
  if (
    e.name === 'CypressError'
    && e.message.includes('timed out')
    && e.message.includes('remote:debugger:protocol')
    && e.message.includes('Network.emulateNetworkConditions')
    && e.message.includes('offline: true')
  ) {
    // do not fail the test
    return false;
  }
});

cy.log('**go offline**')
  .then(() =>
    Cypress.automation('remote:debugger:protocol', { command: 'Network.enable' })
  )
  .then(() =>
    Cypress.automation('remote:debugger:protocol', {
      command: 'Network.emulateNetworkConditions',
      params: {
        offline: true,
        latency: -1,
        downloadThroughput: -1,
        uploadThroughput: -1
      }
    })
  );

This workaround worked for my with cypress open but hangs for cypress run

xdubx commented 2 years ago

same problem on "cypress": "^9.6.1"

weilu commented 2 years ago

Can we please get this working? None of the workarounds above works anymore. I hate to be stuck with version 6.9 because of this

snefa commented 2 years ago

We were also going offline and online in tests and are missing it... +1 here.

asmirnov-style commented 1 year ago

Year 2023 Cypress v10.3.1 Hi there!👋 Issue with going back online still exist.

robertbenthompson commented 1 year ago

Hope everyone had a happy holiday season! As of v12.3.0 looks like the remote:debugger:protocol still fails to go back online. Can anyone else confirm?

Shnrqpdr commented 1 year ago

Hope everyone had a happy holiday season! As of v12.3.0 looks like the remote:debugger:protocol still fails to go back online. Can anyone else confirm?

Yes, unfortunately this issue stills. Any workaround that worked for you?

AlanCodega commented 1 year ago

Cypress 12.6.0, still has the same issue! How do you guys test offline/online in Cypress??

Shnrqpdr commented 1 year ago

Cypress 12.6.0, still has the same issue! How do you guys test offline/online in Cypress??

Pls check my answer (the final answer in the issue discussion) at https://github.com/cypress-io/cypress/issues/235 (i couldn't cite the issue), that workaround worked for me.

xdubx commented 1 year ago

Cypress 12.6.0, still has the same issue! How do you guys test offline/online in Cypress??

@AlanCodega mock all requests

asmirnov-style commented 1 year ago

Kind a way for simulate OFFLINE (while Cypress team fixing 4 y.o bug)

` goOfflineIntercept() {
        cy.log("**go offline**").then(() => {
            cy.intercept(
                {
                    method: "GET",
                    url: "*",
                },
                {
                    statusCode: 404, // if you like and need it
                }
            );
        });
    }

goOnlineIntercept() {
        cy.log("**go online**").then(() => {
            cy.intercept("GET", "*", (req) => {
                req.continue();
            });
        });
    }`
KaiVandivier commented 1 year ago

Thanks @asmirnov-style!

I adapted mine a bit too:

const goOffline = () => {
    cy.log('**go offline**')
        // stub every request with a StaticResponse to simulate network error
        .then(() => cy.intercept('*', { forceNetworkError: true }))
        .then(() =>
            cy.window().then((win) => win.dispatchEvent(new Event('offline')))
        )
}
const goOnline = () => {
    cy.log('**go online**')
        // go back to normal network behavior
        .then(() => cy.intercept('*'))
        .then(() =>
            cy.window().then((win) => win.dispatchEvent(new Event('online')))
        )
}
buffcode commented 1 year ago

Modifying the real onLine attribute, tested in Cypress 12.11.0:

// go offline
cy.window().then((win) => {
  cy.stub(win.navigator, 'onLine').value(false);
  cy.wrap(win).trigger('offline');
});

// go online
cy.window().then((win) => {
  cy.stub(win.navigator, 'onLine').value(true);
  cy.wrap(win).trigger('online');
});
megos commented 9 months ago

Is this issue fixed in v13?

I tried this test and the results are as follows: