Components testing - screenshot causes screen to move causing undesired possible mouse over and other artifacts

[gsouf]

Current behavior

To demonstrate this I created a simple component testing where the component will have a different background color on mouse hover via CSS.

The recorded gif bellow illustrate the issue. After I press the "Run all tests" button then the rendering area of the component will be moved to be under the mouse and the component will receive the hover style of the css before taking the screenshot.

Resulting in the following diff generated by the snapshot plugin (left is expected / right is actual

In some other cases it will capture the cypress ui's tooltip:

Desired behavior

Screenshot to no receive mouse hover or cypress elements

Test code to reproduce

Clone this repository https://gitlab.com/souf/react-webpack-playground/-/tree/cypress-screenshot-blinking using branch cypress-screenshot-blinking.

• Run yarn install and yarn cypress:open
• open the component testing specs "App"
• press the button to run again all test, or to reload the page while the mouse is hover the spot where the dom element will be when "blinking".
• see screenshot in cypress/screenshots

Cypress Version

10.6.0

Node version

16.x

Operating System

linux

Debug Logs

nc

Other

It's a funky one but as bothering as a mosquito 😄

[AtofStryker]

Hey @gsouf . I tried to replicate this issue but didn't get the hover style applied.

I am running on MacOS though. Maybe this is a factor?

[gsouf]

@AtofStryker have you tried to make the left panel smaller in width so that the mouse will be hover the element when it blinks ? In your screencast above the mouse is not hovering the element

[rachelruderman]

Hi @gsouf ! I also tried to reproduce the behavior over 15 runs and was not able to 😿 (see Loom for my panel widths). On a side note, it looks like the package you're using cypress-image-snapshot is no longer maintained, and given the widescale changes in Cypress 10, may no longer be compatible. I see we link to cypress-image-snapshot in the docs, so I'll make sure to pass along the word to update those.

Is it easy for you to reproduce this issue? I.e. out of 10 runs, how often does it happen?

[rachelruderman]

Also, can you try out cypress-plugin-visual-regression-diff? It's compatible with Cypress 10 🎉

[gsouf]

@rachelruderman thanks for your feedback. I can see on your loom example that the element is visually receiving the mouse over and that the spec is failing because there is a diff in the image - have you checked the image saved in cypress/snapshots/?

For me the issue occurs every single times I do that. There is no randomness involved in the process.

I used the package cypress-image-snapshot for quick setup in the reproducible example, but in my real app I'm just using the built-in cy's screenshot function. I'm not sure why I did that in the reproducible example, I probably had a spontaneous need to do something pseudo-fancy 🤷 would be simpler the other way. I'll simplify it tomorrow and I'll try again but I doubt the result will be different.

[gsouf]

@rachelruderman I updated the repo. You can pull and try again.

What I did:

• simplified the spec file to on use cy.screenshot that means you will want to check the screenshot in the directory cypress/screenshots everytime you the spec
• upgraded to cypress 10.6

For me it's failing everytime on both of chrome and firefox linux. Here are the screenshots it took on the last 2 runs I just performed:

[rachelruderman]

Hi @gsouf, this is such a strange one! I created a Linux VM to try to reproduce on FF 99 and Chrome 100 but wasn't able to. You can watch the recording here. I was on node 16.14.2 with

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:    20.04
Codename:   focal

Can you take a look at the video and confirm I'm following the steps correctly? Also, is there anything else different in our environments that crosses your mind? Which browser versions are you testing on? THANK YOU!

[gsouf]

@rachelruderman on the video you sent the issue is reproduced with success.

From the video:

The square is green (mouse hover). It is expected to be red (no mouse hover).

Does that make sense ?

[rachelruderman]

Ohhhhh big derp! @gsouf thank you for the wonderful reproduction repo, your patience and for clarifying! I was on the lookout for the tooltip. I agree, it like you've found a buggo. I'll have this routed to the appropriate team 🙏

Note: I was able to reproduce on MacOS

[jjorgenson-globality]

Is there a workaround for this in the meantime?

[gsouf]

@jjorgenson-globality make sure get your mouse far away from the screenshot area. So when you realize that a screenshot will be captured, you must wake up and move very quickly, crazy I know 🤪

[marktnoonan]

A workaround might be to use the Cypress.Screenshot API to turn off pointer events during a screenshot.

I have not tested this code, but something like this for example seems like it would do the trick:

Cypress.Screenshot.defaults({
  onBeforeScreenshot($el) {
    // turn off pointer events for everything in the body element
    document.querySelector('body').style.pointerEvents = 'none'
  },

  onAfterScreenshot($el, props) {
    // restore default pointer event behavior
    document.querySelector('body').style.pointerEvents = 'initial'
  },
})

This can be declared in your support file. And probably in rare cases initial would not be the right value to restore to, depending on the application.

Even though this has been reported as a component testing issue, I don't see any reason that this couldn't happen from an end-to-end test.

It seems like it might be a good idea for Cypress to turn off pointer events by default during cy.screenshot() though maybe that would have undesirable side effects.

[alycrys]

Any updates on this? I'm experiencing the same issues. "cypress": "^13.0.0", "@frsource/cypress-plugin-visual-regression-diff": "^3.3.10"

[w1nklr]

Any progress here ? This is kind of a blocker issue for component testing in CI/CD :/

BTW, excellent description with video and playground !!