[Bug] headless test fails to click checkbox, headed test has no problems

[ErikSeyferth]

Current behavior:

When I run a e2e test testing a shop flow in headless mode the test fails to click the "accept terms and conditions" checkbox. When I run the test in browser mode it succeeds without problem.

My guess is that cypress fires the click event on checkbox too soon and our JS may not be ready yet to process the click. Adding a cy.wait(300) fixes the problem for now.. but using manual delay's is far from desirable.

the code failing in headless mode:

cy.url().should('contain', '/shop/order/overview/');
cy.get('[cypress-target="privacy_terms_agreed"]').check(); // this check function does not actually check the checkbox
cy.get('[cypress-target="payment-btn"]').click();

I have been experimenting and using cy.get('[cypress-target="privacy_terms_agreed"]').check().uncheck.check(); also makes the test succeed. This reinforces the idea that cypress is to quick with clicking elements in comparison to our code.

I would love to have some insight in why this could be happening and what the best workaround is for problems like these. I understood that the cy.get function should assert an element is ready for interacting before executing any interaction events but it seems to fail in this situation.

Desired behavior:

The headless test has the same behaviour as the headed test

Steps to reproduce:

reproducing is difficult because I think part of the problem here might be the large code base we are testing.

Versions

Cypress 2.1.0 Ubuntu 16 Electron/Electron headless

[jennifer-shehane]

Hi @ErikSeyferth You are correct, adding cy.wait(300) is not ideal. Usually getting to the bottom of these issues involves asserting the state of your application before acting.

The very act of attempting to check the checkbox does assert a lot of the state of the checkbox as outlined here, so I wonder what is 'not ready' when running headlessly.

If I were debugging this, I would add an assertion after the check:

cy.get('[cypress-target="privacy_terms_agreed"]').check();
cy.get('[cypress-target="privacy_terms_agreed"]').should('be.checked')

If this fails, a screenshot should be taken at the point of failure - this will give some insight into what the state of the DOM looks like when it fails. Also - recording and watching the video in the Dashboard helps a LOT when failures happen headlessly if you can set that up.

Let me know where this takes you.

[ErikSeyferth]

Hi @jennifer-shehane thanks for the comment. I did try this assertion and the state of the checkbox is that it is unchecked (in video and screenshot). Why this is the case i cannot discern from the video/screenshot.

The elements are all there and ready for the clicking so to speak :)

[jennifer-shehane]

Oh no. :/ Well, if you do believe this is a bug on our part, we have to some sort of a reproducible example before we can investigate.

[CarrierDirectRoss]

Hi @jennifer-shehane, I am also experiencing an issue when running headlessly where cypress is moving too quick trying to clicking a button that isn't "ready" for interaction yet. This behavior is not present when running headed or adding an arbitrary wait.

[tehandyb]

I'm also finding similar issues, where clicking on an element(react-select component) works in headed mode, but in headless mode I have to be more specific and click on the input element buried in the component or the click won't register.

[kuceb]

this is likely related to #1486, but without a reproducible example I will have to close this issue.

[albertmir]

I can confirm bug is there, even in the Cypress v6.3.0. Only in headless run, and to make things worst, it does not fail always. Just sometimes 😅

After a ton of tests it seems that the bug happens when reaching a page and the first test is the checkbox check. Inverting the tests order for the same page, seem to help preventing the issue and the bug is not happening anymore.

I tried with many selectors, and none of them worked. Even with {force:true} option. This is what I used at last:

cy.get(selector)
    .should('be.visible')
    .should('not.be.checked')
    .check({ force: true })
    .then(($checkbox) => {
      cy.wrap($checkbox)
        .should('be.checked');
    });

Moreover, tests in mochawesome reports are not marked as failed (they were in v4.12.1), but the error screenshot is still being attached.

Also it seems that the bug is causing a run time increase. Compare timings setting the failing test at first place or later. I also tried to replicate one success test result to show you it is not caused by the error screenshot.

[albertmir]

@Bkucera can you re-open it? I think now we have enough information to dig deeper into the bug.

[jennifer-shehane]

@albertmir Can you provide some HTML/CSS/JS to run the tests against to reproduce the issue?

[albertmir]

@jennifer-shehane I think not...