simonsmith / cypress-image-snapshot

Catch visual regressions in Cypress with jest-image-snapshot
MIT License
65 stars 15 forks source link

Different size images: sometimes it's the correct size, sometimes it's twice the size #31

Closed TheColorRed closed 1 year ago

TheColorRed commented 1 year ago

I am not sure if the issue is with cypress or the snapshot tool, but I am trying to do snapshots of an element, and I am getting two different sizes when the image is created/compared. Sometimes the the snapshot is 16x16 (correct) and other times it is 32x32 (incorrect). I am not sure where this issue is coming from.

Here are the settings:

// e2e.js

addMatchImageSnapshotCommand({
  failureThreshold: 0.015, // threshold for entire image
  failureThresholdType: 'percent', // percent of image or number of pixels
  customDiffConfig: { threshold: 0.15 }, // threshold for each pixel
  customDiffDir: '../../dist/cypress/visual_diffs',
  customSnapshotsDir: 'src/integration/snapshots', // relative to each lib, such as /netra/apps/card-e2e/*
  e2eSpecDir: 'src/integration',
  scale: false,
  allowSizeMismatch: true, // avoids false positives when running in jenkins
});

Here is the test (we use cucumber):

Then(`Alias {string} matches snapshot {string}`, (alias, imageName) => {
  if (Cypress.browser.isHeadless) {
    cy.get(`@${alias}`).matchImageSnapshot(imageName);
  }
});

It seems to go into "snapshot mode" run the command, exit "snapshot mode" and then scales up and then takes the actual snapshot that gets saved to disk.

simonsmith commented 1 year ago

Do you get the same problem if you leave all the options as default?

TheColorRed commented 1 year ago

I wonder if this issue is because I am using a MacBook, others seem to be having the same issues...

https://github.com/cypress-io/cypress/issues/6485

TheColorRed commented 1 year ago

So, using the default settings has no effect on the size issue:

image

This is on Github actions:

image

This is my Local:

image

simonsmith commented 1 year ago

Okay, thanks

Are you creating the image in headless mode as well?

simonsmith commented 1 year ago

I wonder if this issue is because I am using a MacBook, others seem to be having the same issues...

Yes, it might well be. I would always advise creating images in the same mode that your tests run. So generate images in headless and then run the tests in that mode as well

In open mode it will use your display DPI

simonsmith commented 1 year ago

Alternatively you can generate them inside a Docker container, which will help with consistency on CI machines. I use this approach

TheColorRed commented 1 year ago

We always generate in headless, but it still uses the MacBook Retina screen to generate the images. I tested this comment:

I don't know why but I am also getting double resolution images when running HEADLESS on a mac. I have to actively set my primary monitor to something that is NOT the built in retina screen to get "normal" sized screenshots.

Doing that fixes the issue, however, it's not something that we (the team) should have to be doing to fix it.

I think I will raise an issue to our team and see what approach we want to take. Thanks!

simonsmith commented 1 year ago

Okay cool. There are many examples of the Docker + Cypress thing but this repo has a basic setup you can use as a guide as well. Mostly around allowing Docker to see your cypress dir on the host machine so it can write the snapshots back to the repo.

Will close for now 👌🏻