cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.04k stars 3.19k forks source link

Cypress v3.3.0 hangs #4247

Closed kweiberth closed 4 years ago

kweiberth commented 5 years ago

Current behavior:

After upgrading to Cypress v3.3.0, I'm experiencing hanging tests, similar to the ones described in https://github.com/cypress-io/cypress/issues/1235 and https://github.com/cypress-io/cypress/issues/1333. The DEBUG=cypress:* logs continuously show the following logs once it hangs:

 cypress:server:timers child sending timer id 28 +6s
  cypress:server:timers child sending timer id 29 +277ms
  cypress:server:video capture stderr log { message: 'frame= 2793 fps= 27 q=13.0 size=    4608kB time=00:01:51.52 bitrate= 338.5kbits/s dup=2099 drop=0 speed=1.09x    ' } +552ms
  cypress:server:timers child sending timer id 30 +119ms
  cypress:server:timers child sending timer id 31 +298ms
  cypress:server:timers child sending timer id 32 +51ms
  cypress:server:video capture stderr log { message: 'frame= 2805 fps= 27 q=12.0 size=    4608kB time=00:01:52.00 bitrate= 337.0kbits/s dup=2103 drop=0 speed=1.09x    ' } +498ms
  cypress:server:video capture stderr log { message: 'frame= 2819 fps= 27 q=13.0 size=    4608kB time=00:01:52.56 bitrate= 335.4kbits/s dup=2108 drop=0 speed=1.09x    ' } +551ms
  cypress:server:timers child sending timer id 33 +1s
  cypress:server:timers child sending timer id 34 +86ms
  cypress:server:video capture stderr log { message: 'frame= 2833 fps= 27 q=12.0 size=    4608kB time=00:01:53.12 bitrate= 333.7kbits/s dup=2118 drop=0 speed=1.09x    ' } +593ms
  cypress:server:video capture stderr log { message: 'frame= 2846 fps= 27 q=12.0 size=    4864kB time=00:01:53.64 bitrate= 350.6kbits/s dup=2122 drop=0 speed=1.09x    ' } +510ms

Desired behavior:

Tests shouldn't hang and all tests should pass. Desired behavior is restored when I revert to Cypress v3.2.0

Steps to reproduce: (app code and test code)

Unfortunately I don't have a chance right now to try and set up a reproducible example. However, v3.3.0 consistently hangs, and v3.2.0 consistently passes.

Versions

Cypress: 3.3.0 Browser: Electron 61

jennifer-shehane commented 5 years ago

Hey @kweiberth, could you provide details about your CI server?

Also, if you could, please try to run the tests using cypress open and choosing the Electron browser with DevTools open - note any errors occurring here for us.

We will definitely need more information here as the information given is not helpful in isolating the problem unfortunately.

noobling commented 5 years ago

I had the same issue today. The whole cypress window hangs e.g. the timer stops. For me I fixed it by rearranging my test that caused cypress to hang and it seem to work fine...

randing89 commented 5 years ago

Same here. If I run it locally the iframe that displays the UI in the middle is completely frozen. The log is like this

  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.9121330451835215, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +1ms
  cypress:server:timers child sending timer id 35 +19s
  cypress:server:timers child sending timer id 37 +774ms
  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.27802936598884, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +1ms
  cypress:server:timers child sending timer id 51 +5s
  cypress:server:timers child sending timer id 53 +384ms
  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.2397979064758744, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +0ms
  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +1ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.23825683881274062, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +1ms
  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.49079102256314777, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +0ms
  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.7071413449160777, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +1ms

The UI won't stuck if I ran the test in Chrome. Here is the log:

  cypress:server:events got request for event: get:project:status, { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests' } +10s
  cypress:server:project get project status for undefined /Users/ran/src/confluence-frontend/next/packages/integration-tests +10s
  cypress:server:project no project id +0ms
  cypress:server:events sending ipc data { type: 'get:project:status', data: { id: 0.6056192172812374, data: { path: '/Users/ran/src/confluence-frontend/next/packages/integration-tests', state: 'VALID' } } } +1ms
  cypress:server:server Getting remote state: { auth: null, props: { port: '443', tld: 'net', domain: 'atlassian' }, origin: 'https://confluence-frontend-fabric-sev1-tests.atlassian.net', strategy: 'http', visiting: false, domainName: 'atlassian.net', fileServer: null } +3s
kweiberth commented 5 years ago

I see a bunch of these errors throughout my tests:

Screen Shot 2019-05-20 at 7 35 00 AM
Error while parsing the 'allow' attribute: 'accelerometer;', 'autoplay;', 'encrypted-media;', 'gyroscope;', 'picture-in-picture' are invalid feature names.

Interestingly, like @randing89 mentions, this has to do with an iframe, specifically a YouTube iframe.

My test suite suite consistently hangs at the start of the fourth describe() block. If I skip the first describe() block, which has these errors, everything runs smoothly. I can also run .only() the fourth describe block, and everything runs smoothly.

Also, during this first describe block, the Cypress UI "hangs" in that the tests pass and the test runner moves on, but the UI would suggest it's still stuck on the first test:

Screen Shot 2019-05-20 at 7 35 23 AM

All of this is happening with Electron 61. Everything runs smoothly with Chrome 74.

Last, this is happening not only on CI like my original post suggested, but locally too. I've updated my original post to remove references to my CI server.

jennifer-shehane commented 5 years ago

@kweiberth I am able to reproduce this console.error

index.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Cypress.io YouTube</title>
</head>
<body>
<iframe width="560" height="315" src="https://www.youtube.com/embed/dr10Z-HpsCQ" frameborder="0"
  allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</body>
</html

test.js

describe('youtube', () => {
  beforeEach(() => {
    cy.visit('index.html')
  })

  it('test', () => {
    expect(true).to.eq(true)
  })
})

Open the DevTools console to see the console errors:

Screen Shot 2019-05-20 at 9 22 12 PM

On initial thought, I thought it had something to do with Chrome's Autoplay policy changes that maybe some of it took effect in Chromium 61, but I don't know why these errors would then disappear when running the tests in Chrome 74. Plus it doesn't look like much of this was implemented for Chromium 61.

Am going to do some more research.

jennifer-shehane commented 5 years ago

Also, I'm not sure if the issue @kweiberth described is the same 'hanging' issue everyone else is having. Can anyone confirm if they are using video embedded in iframes? If not, we should probably move these autoplay problems to a new issue.

beepboopitschloe commented 5 years ago

My tests are also hanging in CI, but I'm not using video or iframes.

immunity20 commented 5 years ago

Same problem here. with Electron 61. I have 5 integrations tests (1st, 2nd and 3rd are similar) 3rd test hang waiting for xhr response (although the server shows the 200

that xhr request is requested (with slightly different variables) multiple times in all tests. but for some reason Electron 61 (headless or window) hangs always in the same particular request xhr (which it already has sent ~8 more in the same test) . If i try to run the test (while Electron 61 is hanging) with chrome, the Electron 61 start to respond again (When chrome browser does the same request)

huyvanvu commented 5 years ago

Same problem. Failed at Electron 61 but work well with Chrome. I don't have either video embedded or iframes. After trying these steps, it's now working for me.

  1. Delete folder Cypress in C:\Users\***\AppData\Roaming. Where *** is your username.
  2. Delete node_modules
  3. Run yarn
jennifer-shehane commented 5 years ago

We'll need something we can run to reproduce the hanging issue. Can't debug this without that.

rafael-anachoreta commented 5 years ago

My case has been the opposite - Chrome will sometimes hang whereas Electron never seems to. I've had this happen on 3.1.0, 3.2.0 and 3.3.1.

I don't have a reproducible example I can share at the moment other than my DEBUG logs, which I would rather share privately. @jennifer-shehane if you think those could help let me know and please advise how I could share them.

mpahuja commented 5 years ago

@jennifer-shehane What additional information would be helpful here? We have the same issue when running on Linux (Ubuntu) containers but the same code works fine on Mac. We are running tests on Chrome and have this issue with Cypress: 3.2.0

ghost commented 5 years ago

Hello everyone. I've been seeing this issue for a while now (originally reported it in Aug '18 https://github.com/cypress-io/cypress/issues/2377) and haven't been lucky to figure it out. Though yesterday I found out that when video encoding seems to hang it's in fact due to browser not being killed properly.

Just after test finishes I see following in debug:

cypress:server:run attempting to close the browser
cypress:server:browsers killing browser  

followed by request to tracking scripts that are fired on "unload" event from JS.

And what I've noticed is, that when the request is fired (browser is not killed fast enough?) in video it looks like that:

When the request isn't fired (and only then) I receive cypress:server:browsers browser process killed in the debug log.

Edit:

I've just figured out a PoC to behaviour I've described, which is also consistent with problem I have on my page:

index.html:

<html>
<head>
    <script>
        window.onbeforeunload = function (e) {
            e = e || window.event;

            if (e) {
                e.returnValue = 'Sure?';
            }

            return 'Sure?';
        };
    </script>
</head>
<body>
<div id="test">test</div>
</body>
</html>

test.js

describe('poc', function () {
    it('should hang | @olxpl @olxua @olxkz @olxbg @olxpt', function() {
        cy.visit('/index.html');
        cy.get('#test').should('exist');
    });

});

Can't believe I didn't notice that earlier but we do, in fact, have a prompt before closing the window on last page the test is visiting.

Here's a video https://www.youtube.com/watch?v=2GjGaJI93Ow that's being recorded.

@jennifer-shehane please have a look

zsalzbank commented 5 years ago

Something similar to the above is happening for me as well. When I run with 3.2.0, everything is working as it should. Seems unrelated to onbeforeunload for me (we have one, but I get the same results when commenting out the code).

When running 3.3.2, the left side of my screen (with all the steps and assertions) eventually stops working, even though the test is continuing in the browser side of the screen. After the test completes (and passes), further tests are not run anymore.

gabbersepp commented 5 years ago

Not sure if this helps anyone: We also run into issues with hanging builds. To see what is going on and what the test is doing at this time, we activated video recording. After that, there were no hanging builds anymore...

jennifer-shehane commented 5 years ago

@tkociemba-olx Your issue is a known issue here: https://github.com/cypress-io/cypress/issues/2118

noobling commented 5 years ago

Thanks Jenny

On Thu, 18 Jul 2019 at 2:13 pm, Jennifer Shehane notifications@github.com wrote:

@tkociemba-olx https://github.com/tkociemba-olx This issue is a known issue here: #2118 https://github.com/cypress-io/cypress/issues/2118

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cypress-io/cypress/issues/4247?email_source=notifications&email_token=AFRBXUFO7PG7YHWTTFGSINLQAACXRA5CNFSM4HN5WJEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HN3TQ#issuecomment-512679374, or mute the thread https://github.com/notifications/unsubscribe-auth/AFRBXUHC7A6UMQYAMOIEXYDQAACXRANCNFSM4HN5WJEA .

--

Regards, David

sarink commented 5 years ago

This happens in my docker container on 3.4 as well... is there any update on a workaround or fix?

sarink commented 5 years ago

We chewed through like 3 hours of build minutes in my CI server because of this before anyone realized :( Running in the local macos gui, everything works fine, but running: npm ci && cypress install && cypress open from the cypress/browsers:node12.6.0-chrome77 docker container leads to the same hanging error that @randing89 has https://github.com/cypress-io/cypress/issues/4247#issuecomment-493880570 (which, now that I'm looking more carefully, actually appears to be a very different error from the OP)

ManfredHu commented 5 years ago

same problem happen on CentOS7.

run e2e test by vue-cli-service script

"test:e2e:headless": "vue-cli-service test:e2e -- --headless"
jennifer-shehane commented 4 years ago

The original issue of seeing this warning in the console has been fixed since Cypress 3.5.0 when we updated Electron as part of https://github.com/cypress-io/cypress/pull/4720

Error while parsing the 'allow' attribute: 'accelerometer;', 'autoplay;', 'encrypted-media;', 'gyroscope;', 'picture-in-picture' are invalid feature names.

Closing as resolved.