Test runner window when occluded affects test results

[OlsonDev]

Current behavior

We have a "global loading" element in our app that covers the entire screen any time a component fetch()es data (well, it's more sophisticated than that, but it doesn't matter). I've noticed that if I put another window on top of Cypress's "Test runner" window, my tests will consistently fail, claiming something can't be clicked because it's being covered by another element (our "global loading" element). The same tests will consistently pass if the window isn't covered, even if it's only a few pixels that are exposed.

I believe this is Chromium detecting it's fully covered and throttling itself somehow. However, I don't think it's only slowing its rate of processing things, I think it's fully halting things. If I add a { timeout: 10000 } to my commands, they still fail when the window is covered.

I can also reproduce this by minimizing the window.

I've played around with covering the window with a lot of different apps -- because at first I couldn't make it fail when SQL Server Management Studio was covering it up, but now I can't even reproduce that. That is, SSMS covering Cypress consistently fails the test now.

Desired behavior

Cypress test runner window state should not affect the results

Test code to reproduce

I tried to put together a simple repro, forking cypress-test-tiny, but of course it passes regardless of windows covering it or being minimized. I'll have to come back to this; but I'm still submitting this now in case anyone has ideas.

Versions

6.0.0, but probably earlier and 6.1.0

[jennifer-shehane]

What version of Chrome is this occurring in? Have you tried running in another browser (like Firefox) to isolate it to a browser issue?

This makes me think that perhaps Chrome 87's Occlusion Tracking is affecting the way your app is performing. https://blog.chromium.org/2020/11/tab-throttling-and-more-performance.html

[OlsonDev]

I'm using Chrome Version 87.0.4280.88 (Official Build) (64-bit).

Good idea; using other browsers: Changing Cypress to use Firefox 83.0 (64-bit) -- consistently passes.

Changing Cypress to use Microsoft Edge Version 87.0.664.57 (Official build) (64-bit) -- consistently fails like Chrome.

Changing Cypress to use the bundled Electron, I believe it's v11.0.2 (there's no "About Electron", so I just inspected window.navigator.userAgent) -- consistently passes. I'm guessing eventually this will not be the case -- that Electron just doesn't have the Occlusion Tracking fully integrated/enabled yet?

I knew Chrome had some sort of system like that; I didn't realize it was released in M87. I feel like I've been getting these errors randomly (prior to tracking down the cause) a while ago, but it's hard to be sure. 🤷🏻‍♂️

[jennifer-shehane]

Actually it seems this has been in a few versions of Chrome, but is being expanded to windows. Can you try disabling it to see if it improves the behavior:

// cypress/plugins/index.js
module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, launchOptions) => {
    if (browser.family === 'chromium' && browser.name !== 'electron') {
      launchOptions.args.push('--disable-backgrounding-occluded-windows')

      return launchOptions
    }
  })
}

I opened a PR here regardless because we should be disabling this https://github.com/cypress-io/cypress/pull/9640

[OlsonDev]

Pretty sure you nailed it! I added it, it worked; I commented it out and relaunched the test, and it didn't work as expected; uncommented and relaunched, it worked again.

Thank you so much!

[jennifer-shehane]

The code for this is done in cypress-io/cypress#14325, but has yet to be released.

We'll update this issue and reference the changelog when it's released.

[cypress-bot[bot]]

Released in 6.2.1.

This comment thread has been locked. If you are still experiencing this issue after upgrading to Cypress v6.2.1, please open a new issue.