meinaart / cypress-plugin-snapshots

Plugin for snapshot tests in Cypress.io
MIT License
497 stars 117 forks source link

Different Actual and Reference viewports between cypress open and run(bug) #71

Closed Kamilius closed 4 years ago

Kamilius commented 4 years ago

Describe the bug I was developing a snapshots for all of my active pages (for visual regression testing automation) using UI $ cypress open and viewport set to 1440x900 with cy.viewport('macbook-15'); on a global level.

When tried to run everything headless using cypress run, started getting errors everywhere. Diff shown me, that actual image snapshot differs from expected via viewport size. After checking actual resolution of generated images, got following results:

Tried already defining a viewport width/height explicitly inside cypress.json or even passing as CLI parameters. None of them helped.

To Reproduce Steps to reproduce the behavior:

  1. Create a beforeEach() block inside existing test which uses cy.document().toMatchImageSnapshot();
  2. Put cy.viewport(1440, 900); inside of this beforeEach() block
  3. Initiate test run and take reference image snapshot with cypress open (UI version of Cypress)
  4. After test success. Make sure, image size of taken reference image, corresponds to 1440x900px
  5. Run test again, but this time on Electron with cypress run
  6. Test will fail and generate two additional files diff and actual
  7. Check actual image dimensions and you'll see that it's 1280 × 720 for some reason, ignoring viewport configuration.

Same steps above can be repeated, while moving beforeEach() { cy.viewport(1440, 900); } into global commands.js, or even specifying them inside cypress.json or passing as CLI parameters into cypress run.

Expected behavior Images taken both during cypress open or cypress run are respecting custom viewport configuration. In short there should be no difference in dimensions between images taken under cypress open and cypress run.

Screenshots Campaign Insight  Paused tab  Placeholder #0 actual Campaign Insight  Paused tab  Placeholder #0 diff Campaign Insight  Paused tab  Placeholder #0

Desktop (please complete the following information):

zac-robinson commented 4 years ago

I am experiencing this same issue using Cypress 3.7.0 and cypress-plugin-snapshots 1.2.9

meinaart commented 4 years ago

This is more a Cypress bug. Not sure if I can fix this.

linukkis commented 4 years ago

I'm pretty sure it's not an issue of this plugin, but if anyone still has this problem, here's how I managed to solve it based on what I found at https://github.com/cypress-io/cypress/issues/2102 , which is basically giving custom window size arguments to the browser upon launch:

// cypress/plugins/index.js

const { initPlugin } = require('cypress-plugin-snapshots/plugin');

module.exports = (on, config) => {
    initPlugin(on, config);
    on('before:browser:launch', (browser = {}, args) => {
        if (browser.name === 'electron') {
            args.width = 1920;
            args.height = 1080;
            return args;
        }
        if (browser.name === 'chrome') {
            args.push('--window-size=1920,1080');
            return args
        }
    });
    return config;
};

In my case I set it to 1920x1080, but it was necessary to put the event handler after initPlugin(on, config);. If I put the handler before that line, the custom arguments would not be pushed.

ivoiv commented 4 years ago

It's an issue with Cypress. Check here

https://github.com/cypress-io/cypress/issues/2102