search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.09k stars 3.19k forks source link

Simulate offline mode #235

Open bahmutov opened 8 years ago

bahmutov commented 8 years ago

Description Offline support is becoming important to web applications. It would be nice to switch the browser to the offline mode from Cypress either by test or for all tests and run it. If might be difficult and might require Cypress to handle ServiceWorkers very well.

Additional Info

brian-mann commented 8 years ago

This should be possible to do, but it's not immediately obvious or easy.

Thoughts / Concerns

I'll move this into our Roadmap for further research.

jennifer-shehane commented 7 years ago

With Chrome 63 coming out we will be able to support the debugger protocol. #832

bbsimonbb commented 6 years ago

Good onya guys. I'm ready for this right now. You know what to do.

juristr commented 6 years ago

Is there any progress/news on this? Would be cool to be able to test PWAs 👍

jennifer-shehane commented 6 years ago

No progress on this unfortunately.

maku-zuhlke commented 6 years ago

I am using Redux and we store the online/offline status of the browser there. So what we did was to expose the Redux store like this in eg index.js (only works on localhost):

if (window.Cypress) {
    window.store = store
}

Then in a test in cypress we dispatched an action to set the Redux state to be offline:

cy.window().its('store')
  .then(
    store => store.dispatch({type: 'OFFLINE', status: true})
  );
lolpez commented 5 years ago

Guys it's 2019 👀

mhrisse commented 5 years ago

+1

jennifer-shehane commented 5 years ago

Related issue for clearing ServiceWorkers: https://github.com/cypress-io/cypress/issues/702

@lolpez We prioritize issues we work on based on a variety of factors - including the number of users requesting/helped by a feature. While this is still on our Roadmap, many other features are a higher priority today as a result of this assessment. Sorry for the delay.

gr2m commented 4 years ago

I tried to simulate offline by setting up a reverse proxy to my running server and then shutting it down.

const httpProxy = require("http-proxy");

context("Offline", () => {
  before(() => {
    httpProxy.createProxyServer({ target: "http://localhost:9000" }).listen(9001);

    cy.log("Open / on proxy");
    cy.visit("http://localhost:9001/");
  });

  it("should show update notification", () => {
    // wait for initial service worker installation
    cy.wait(3000);

    // stop the proxy server
    proxy.close();

    // reload the page
    cy.reload();
  });
});

But I get a http.createServer is not a function error.

If I run just these two files in a separate file there is no problem

const httpProxy = require("http-proxy");
httpProxy.createProxyServer({ target: "http://localhost:9000" }).listen(9001);

Is this a known limitation / problem?

barnabynorman commented 4 years ago

It would be really good to have this feature.

I can currently set the offline state in our vuex store but when changing pages (ie cy.visit) the application rechecks the state and updates the store value and is flushing queues before I get a chance to set the store again!

miroslavvojtus commented 4 years ago

Absence of this feature actually forces us to use selenium with direct access to driver api so we can test our core functionality in offline mode as our use case is offline-first.

I hope you will find time soon for this.

Ozzyjtjustin commented 4 years ago

The project my team is working on requires that we are able to simulate a No Internet Connection environment as well.

Please provide an update on this!

jpita commented 4 years ago

hello, any updates on this? is it on the roadmap?

jennifer-shehane commented 4 years ago

This issue is still in the 'proposal' stage, which means no work has been done on this issue as of today, so we do not have an estimate on when this will be delivered.

shubhsherl commented 4 years ago

One way of simulating offline mode: 

cy.server({ force404: true }); //  offline mode

cy.server({ enable: false});  // online mode

https://docs.cypress.io/api/commands/server.html

jpita commented 4 years ago

One way of simulating offline mode:

cy.server({ force404: true }); //  offline mode

cy.server({ enable: false});  // online mode

https://docs.cypress.io/api/commands/server.html

that only blocks the calls, it doesn't tell the browser there's no internet

shubhsherl commented 4 years ago

that only blocks the calls, it doesn't tell the browser there's no internet

Yes, it doesn't tell the browser that there is no internet, but by preventing the requests, we can test the offline functionality of PWA. It's just a workaround.

jpita commented 4 years ago

In my case it doesn't help, it should appear a "offline" banner and it doesn't. Thanks anyway

EtienneBruines commented 4 years ago

The root of the app is not cachable because of the MITM that Cypress performs (`document.domain = 'localhost').

This would make it (at least to me) that one would enable offline-mode after having started testing. First you'd want to install your service worker and such, adding all required files to the cache, and then run some tests. This would mean a global option is not required/desired - if you have no cache, no prior visits to the site - offline will never work anyways.

Design

// Quick and dirty
cy.offline()
// - do tests inbetween
cy.online()

// A probably better way:
cy.network({offline: true})
// - do tests inbetween
cy.network({offline: false])

The latter would enable Cypress to at at later point also add throttling in the options. Not sure as to the usefulness of throttling, but that's point of discussion for another issue.

Docs

The Chrome Debugger (Chrome, Electron) allows for the following, but Firefox does not. 

    Cypress.automation('remote:debugger:protocol', {
        command: 'Network.enable',
      })

      Cypress.automation('remote:debugger:protocol', {
        command: 'Network.emulateNetworkConditions',
        params: {
          offline: options.offline,
          'latency': 0,
          'downloadThroughput': 0,
          'uploadThroughput': 0,
          'connectionType': 'none',
        },
      })

Hurdles

@brian-mann wrote: If we did use Chrome's APIs to achieve this, it would essentially break Cypress's ability to do automation since websocket events would no longer be able to reach the server. So if we did tap into Chrome we could only partially throttle and not fully force into offline mode. Another option would be to use the WebRequest API's and whitelist Cypress network traffic but throttle or reject others.

Half-way through my test (within a single it), I was able to enable offline-mode without any problems (manually, that is). After the tasks have been queued at the start, there should not be any issues with that.

Update

I've created a PR, but I'm not making any progress on Firefox.

According to this issue Firefox does not seem to implement Network.emulateNetworkConditions yet - and I haven't found a suitable replacement either. For some reason, those APIs don't have easily accessible documentation.

The Firefox UI has a Work offline mode, which does the same - I just haven't found a way to automate this.

Update 2

Unless we figure out how to automate it for Firefox, the chances of any PR making it into Cypress are slim to none.

Call to everyone: feel free to try and find a way ;-).

Update 3

Work on the Network.emulateNetworkConditions CDP command in Firefox has started. All seems to be going well, except that Firefox - as of writing- , allows access to everything that resolves to localhost even in Work Offline-mode. This is still being figured out to make automated testing a bit more useful.

lolpez commented 4 years ago

Guys it's 2020 👀

jpita commented 4 years ago

Guys it's 2020

Your point?

rpocase commented 4 years ago

@EtienneBruines Any chance this could take a staged approach with chromium based support first? There are already other examples in the docs of features/workarounds that are only available in chrome, and some coverage is better than none.

EtienneBruines commented 4 years ago

@rpocase if there is enough demand for it, I guess it could.

As mentioned in #6932 we could publish it as a separate npm package (open source, so feel free, even if I don't end up doing so). The Cypress team has expressed (rightfully so) to not want this 'half-baked', so either full support in all by-Cypress-supported-browsers, or as a separate npm package that people can use at own risk.

Today we submitted a PR to https://bugzilla.mozilla.org/show_bug.cgi?id=1553849 to enable Network.emulateNetworkConditions on Firefox as well. The people working there really were an awesome help in getting it this far.

I can't speak on behalf of Cypress (just an outsider here :sweat_smile: ), but I'm guessing that once Firefox support lands (hopefully Firefox 78.xxxx, but I don't know for sure), and I add those tests to #6932 - they'd be up to including it.

flotwig commented 4 years ago

Today we submitted a PR to bugzilla.mozilla.org/show_bug.cgi?id=1553849 to enable Network.emulateNetworkConditions on Firefox as well. The people working there really were an awesome help in getting it this far.

@EtienneBruines awesome! we have been closely monitoring Firefox team's progress on the CDP implementation, we are excited to use it in Cypress

I can't speak on behalf of Cypress (just an outsider here sweat_smile ), but I'm guessing that once Firefox support lands (hopefully Firefox 77.xxxx, but I don't know for sure), and I add those tests to #6932 - they'd be up to including it.

I believe CDP support is still only available in Firefox Nightly releases, so we can't really use any of it until they make it available in all editions of Firefox. Never mind, just learned that CDP support is in all editions, so this may be possible if it makes it to stable. Perhaps we'll also be able to switch some of our other automation code over to using CDP at the same time.

EtienneBruines commented 4 years ago

Quick update:

With all of the above patches in Firefox, I've managed to successfully implement cy.network on Firefox as well. Offline-only, so no throttling etc., but hey - that's all this issue is about.

Regarding Chrome there's still the issue of service-workers ignoring offline emulation, but for the past two weeks others have been working hard on Chrome to fix that issue as well - so I'm not worrying about that, that just means that service-worker support is only for newer Chrome versions. Seems like that Chrome-issue was fixed this month :tada:

Updated the PR with the progress - still lots of work to be done :sweat_smile:

jpita commented 4 years ago

@EtienneBruines is cy.network already available in the API? if yes, can u point me to the docs? haven't been able to find any info.

EtienneBruines commented 4 years ago

@jpita The underlying support for such a feature only just landed in Firefox Nightly, and Chrome is still working on some important bugfixes.

That said, the Pull-Request (that is, #6932) was updated - but the PR has not yet been merged into cypress/develop and is therefore also not (yet) a part of any Cypress release. Sorry for the confusion.

The cy.network I mentioned is the one hypothesized in https://github.com/cypress-io/cypress/issues/235#issuecomment-608367340

flotwig commented 4 years ago

Offline-only, so no throttling etc., but hey - that's all this issue is about.

FYI, #4176 will implement offline mode as well as simulated throttling/latency at the HTTP proxy level, but it's still a ways off. So having "offline mode" accessible via a plugin that uses CDP to get the job done will still be valuable in the mean time.

EtienneBruines commented 4 years ago

@flotwig That's nice to know.

I'm just wondering, will #4176 affect the navigator.onLine representations, among others? Or is it a fancy way of turning the Cypress-Proxy off? (I don't know what the navigator.onLine value does when the proxy is offline.)

The CDP-Network.emulateNetworkConditions acts like an "airplane mode"-toggle, and therefore does influence that value (and fire the events that come with it) - useful for testing showing a message "You are currently offline" to the user.

flotwig commented 4 years ago

@EtienneBruines From https://developer.mozilla.org/en-US/docs/Web/API/NavigatorOnLine/onLine, it sounds like all browsers return true for navigator.onLine if the OS has any active network connection, regardless if they truly have access to the Internet or not.

Since the browser will still be able to see the OS's network connection regardless of what the proxy layer is doing in #4176, I'm assuming #4176 will not affect navigator.onLine.

The HTML spec is pretty candid about this point:

This attribute [navigator.onLine] is inherently unreliable. A computer can be connected to a network without having Internet access. — https://html.spec.whatwg.org/multipage/offline.html#navigator.online

AdrianoFerrari commented 4 years ago

I'm also trying to test a PWA, and being able to manually set the test browser to offline is essential to me.

I've gotten close, using this:

cy.route2('*', (req) => {req.destroy()})

Or, as an alternative:

cy.route2('*', {forceNetworkError: true})

(note: route2 is experimental and needs to be enabled manually)

The issue in both cases is that, while the requests are indeed dropped and do fail, they are also constantly being retried (see below). Any ideas? Would this get us close to testing offline modes?

Screenshot_2020-11-10_10-29-18

EtienneBruines commented 4 years ago

@AdrianoFerrari Which Cypress version is that? It seems like https://github.com/cypress-io/cypress/issues/8679, which should* be fixed in 5.6.0

*haven't verified this myself, but I trust the Cypress team.

AdrianoFerrari commented 4 years ago

I was using 5.3.0. Confirmed that with 5.6.0 I don't get the repeated/rapid retries of the image downloads. But the other failed requests (e.g. to db/) also retry.

Closer, but I'm not sure how to proceed. Will post back with more details when I can.

bahmutov commented 3 years ago

We can do the offline mode during the test using Chrome debugger protocol, read https://www.cypress.io/blog/2020/11/12/testing-application-in-offline-network-mode/

EtienneBruines commented 3 years ago

@bahmutov Do you know what the status is of exposing the Chrome debugger protocol for Firefox instances? The ball is in Cypress' court regarding that.

bahmutov commented 3 years ago

Firefox has certain limitations there, so that is still something we have not done

Sent from my iPhone

On Dec 1, 2020, at 09:58, Etienne Bruines notifications@github.com wrote:

 @bahmutov Do you know what the status is of exposing the Chrome debugger protocol for Firefox instances? The ball is in Cypress' court regarding that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

lolpez commented 3 years ago

Guys it's 2021 👀

jennifer-shehane commented 3 years ago

Reopening since this is not supported cross browser within the Cypress product.

There is a workaround outlined here for Chromium browsers: https://www.cypress.io/blog/2020/11/12/testing-application-in-offline-network-mode/

Jacob-McKay commented 3 years ago

Would really like to adopt the CDP solution, but it appears to only work when I target localhost in my test. I verified this behavior with the cypress sample, am I crazy? https://github.com/cypress-io/cypress-example-recipes/issues/721

I'm going to try the fallback of mocking out window.navigator.onLine in the meantime

jennifer-shehane commented 3 years ago

You can use cy.intercept('*', { forceNetworkError: true }) to also mimic offline mode in some circumstances, so that is something to explore.

Kniaziev commented 3 years ago

This one => https://www.cypress.io/blog/2020/11/12/testing-application-in-offline-network-mode/ is not actual anymore, since last month, it started to fail after going offline, can't get back online, and all the next tests fails because it is still offline.

SimonLandeholm commented 2 years ago

The workaround now fails in chrome 96, when you wait intercepts with routeHandlers the request gets stuck in "pending" after going online again. I've tried cypress 8 and 9.

archfz commented 2 years ago

@lopezc184 I was able to reproduce the issue, only on chrome and edge. This doesn't seem to be an issue with cypress-terminal-report plugin. For example if you uninstall the plugin and add the following to your code, you will still reproduce the issue:

plugins.js

module.exports = (on) => {
  on('task', {
    'mytask': () => console.log('lol'),
  });
};

end of your spec (the same code as you have above)

   // ...
   cy.wrap(null).then(() => {
      goOnline()
      assertOnline()
      cy.get('some element').should('be.visible')
    })
   cy.task('mytask');

The issue seems to be with how cypress is communicating with the node js backend. Seemingly it uses network calls and messing with the network connection breaks it. That is what I think is happening, but could be other cause as well.

jhnns commented 2 years ago

For documentation: The debugger protocol approach as described on the blog doesn't work since 7.3.0. This is probably the cause.

myasul commented 2 years ago

I have implemented the following but nothing seems to work. Code can be seen below. 

export class NetworkConnectivity {
    private static readonly affectedUrlsRegex = /(?<=localhost:\d+)\/(apollo|api)\/.+$/g

    public static goOffline () {
        const offlineDevtoolCommand = {
            command: 'Network.emulateNetworkConditions',
            params: {
                offline: true,
                latency: -1,
                downloadThroughput: -1,
                uploadThroughput: -1
            }
        }

        return cy
            .then(() => Cypress.automation('remote:debugger:protocol', { command: 'Network.enable' }))
            .then(() => cy.task('sendCommandsToServiceWorker', { command: 'Network.enable' }))
            .then(() => cy.task('sendCommandsToServiceWorker', offlineDevtoolCommand, { timeout: 120000 }))
            .then(() => Cypress.automation('remote:debugger:protocol', offlineDevtoolCommand))
            .then(() => {
                // Reference: https://docs.cypress.io/api/commands/intercept#Controlling-the-response

                // @ts-ignore
                Cypress.config('network', { isOnline: false })

                cy.intercept(this.affectedUrlsRegex, req => {
                    // @ts-ignore
                    const { isOnline } = Cypress.config('network')

                    if (!isOnline) {
                        req.destroy()
                    }
                })
            })
    }

Both of these would not entirely intercept the request and the request would still return a 200 response. This might be because we are using service workers. To try and fix this issue, I tried using puppeteer-core to access the active service workers and send out the commands. This worked when tested locally but fails in CI/CD with the error The Test Runner unexpectedly exited via a exit event with signal SIGSEGV but without further explanations about the failure.

const puppeteer = require('puppeteer-core')

const REMOTE_DEBUGGING_PORT = 9222

async function sendCommandsToServiceWorker ({ command, params }) {
    console.log(`Plugin started executing command ${command}, params: `, params)

    const browser = await puppeteer.connect({ browserURL: `http://localhost:${REMOTE_DEBUGGING_PORT}` })
    const targets = await browser.targets()
    const serviceWorkers = targets.filter((t) => t.type() === 'service_worker' || t.type() === 'other')

    const sendPromises = serviceWorkers.map(async (serviceWorker) => {
        const cdpSession = await serviceWorker.createCDPSession()
        await cdpSession.send(command, params)
    })

    await Promise.all(sendPromises)

    return null
}

module.exports = {
    sendCommandsToServiceWorker
}

Here is the screenshot of the CI/CD error. image

I would really appreciate if anyone has solved this issue or have a workaround when using service workers.

lolpez commented 2 years ago

Guys it's 2022 👀

aktibaba commented 2 years ago

This worked for me

cy.log('go offline'); return Cypress.automation('remote:debugger:protocol', { command: 'Network.enable', }).then(() => { Cypress.automation('remote:debugger:protocol', { command: 'Network.emulateNetworkConditions', params: { offline: true, latency: -1, downloadThroughput: -1, uploadThroughput: -1, }, }) return cy.wait(1000); });

laerteneto commented 2 years ago

I can go offline, but I am not able to get back online again. So, following test cases fail.

However, it goes online/offline without any problems on Electron, but on Chrome and Edge it does not.

image

aktibaba commented 2 years ago

Copy exactly same from above and try again