meinaart / cypress-plugin-snapshots

Plugin for snapshot tests in Cypress.io
MIT License
497 stars 118 forks source link

(bug) screenshots are differents between local CI #150

Closed tonai closed 4 years ago

tonai commented 4 years ago

Describe the bug I got 3 tests that are fine when they ran locally, but when I push to CI (using CircleCI) all my tests fail.

In fact They are ok with default threshold, but they are ko when I use:

    cy.document().toMatchImageSnapshot({
      imageConfig: {
        threshold: 0.001,
        thresholdType: 'percent'
      }
    });

To Reproduce Here is the repo: https://github.com/tonai/react-deploy

Expected behavior I would like my tests to pass on CI environment.

Desktop

CI seems to run on Debian 10.5

Additional context Electron version is the same in both environment.

When running cypress locally I got following informations:

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:      5.2.0                                                                            │
  │ Browser:      Electron 83 (headless)                                                           │
  │ Specs:        2 found (e2e.js, ui.js)                                                          │
  │ Experiments:  experimentalFetchPolyfill=true                                                   │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

And on CI I have following informations:

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:      5.2.0                                                                            │
  │ Browser:      Electron 83 (headless)                                                           │
  │ Specs:        2 found (e2e.js, ui.js)                                                          │
  │ Params:       Tag: false, Group: false, Parallel: false                                        │
  │ Run URL:      https://dashboard.cypress.io/projects/49dchw/runs/8                              │
  │ Experiments:  experimentalFetchPolyfill=true                                                   │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
tonai commented 4 years ago

Ok I tried to generate the screenshots locally with Debian 10 and it finally works.