jaredpalmer / cypress-image-snapshot

Catch visual regressions in Cypress
MIT License
888 stars 160 forks source link

First run return "Error: Image was NaN% different from saved snapshot with undefined different pixels. See diff for details: undefined" #136

Closed ratonjuancarlos closed 4 years ago

ratonjuancarlos commented 4 years ago

Hi everyone First of all, thank for your great job. We are trying this plugin and give us exactly what we need... until now :)

Preconditions

  1. CRA (Create react app) fresh installed with npx
  2. cypress installed (version 4.9.0)
  3. cypress-image-snapshot installed (version 3.1.1)
  4. addMatchImageSnapshotCommand added to cypress/support/commands.js without any parameter. Just use the default behavior. (I've tried also with the options given in the README and get the same result)
  5. addMatchImageSnapshotPlugin added to cypress/plugins/index.js
  6. test created calling cy.matchImageSnapshot('root'); in cypress/visual.spec.js
  7. script "cy:run":"cypress run" added in package.json
  8. My computer settings are: 8.1 OS: Debian 9 8.2 npm --version: 6.13.4 8.3 node --version: 12.10.0

Steps to reproduce

  1. In one bash terminal: npm start to run the react app
  2. In another bash terminal: npm run cy:run 2.1 Also tried to run previously npm run cy:open to create the cypress folder and the cypress.json config file

Actual and Expected result

Expected behavior

When the test run, cypress-image-snapshot shoudl check if a previous screenhost has been taken, As for the first run, it will see there is not previous screenshot, so it will create a new one. The test shouldn't fail, becasue there is no previous screenshot to compare.

Actual behavior:

The test fails with a message:

- Error: Image was NaN% different from saved snapshot with undefined different pixels. See diff for details: undefined

But also, if the test is failing, the plugin should create a cypress/snapshots/__diff_output__ folder with. This folder is not created when the test fails.

My files

support/commands.js

import { addMatchImageSnapshotCommand } from "cypress-image-snapshot/command";

addMatchImageSnapshotCommand();

cypress/plugins/index.js

const {
  addMatchImageSnapshotPlugin,
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config);
  on('after:screenshot', (details) => {
    console.log(details) // print all details to terminal
  })
}

cypress/visual.spec.js

describe('test', () => {
    it('trying snapshots', () => {
      cy.visit('http://localhost:3000');
      cy.matchImageSnapshot('root');
    });
  });

package.json

{
  "name": "cra",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^4.2.4",
    "@testing-library/react": "^9.5.0",
    "@testing-library/user-event": "^7.2.1",
    "cypress": "^4.9.0",
    "cypress-image-snapshot": "^3.1.1",
    "react": "^16.13.1",
    "react-dom": "^16.13.1",
    "react-scripts": "3.4.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "cy:run": "cypress run",
    "cy:open": "cypress open"
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

Log


> cra@0.1.0 cy:run /home/pablo/projects/legion etrangere/projets/cra
> cypress run

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    4.9.0                                                                              │
  │ Browser:    Electron 80 (headless)                                                             │
  │ Specs:      1 found (visual.spec.js)                                                           │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  visual.spec.js                                                                  (1 of 1)

  test
{
  size: 30423,
  takenAt: '2020-07-04T17:22:34.383Z',
  dimensions: { width: 1000, height: 660 },
  multipart: false,
  pixelRatio: 1,
  name: 'root',
  specName: 'visual.spec.js',
  path: '/home/pablo/projects/legion etrangere/projets/cra/cypress/screenshots/visual.spec.js/root.png',
  scaled: false,
  blackout: [],
  duration: 299
}
{
  size: 130004,
  takenAt: '2020-07-04T17:22:34.707Z',
  dimensions: { width: 1280, height: 720 },
  multipart: false,
  pixelRatio: 1,
  specName: 'visual.spec.js',
  testFailure: true,
  path: '/home/pablo/projects/legion etrangere/projets/cra/cypress/screenshots/visual.spec.js/test -- trying snapshots (failed).png',
  scaled: true,
  blackout: [],
  duration: 355
}
    1) trying snapshots

  0 passing (2s)
  1 failing

  1) test
       trying snapshots:
     Error: Image was NaN% different from saved snapshot with undefined different pixels.
See diff for details: undefined
      at Context.eval (http://localhost:3000/__cypress/tests?p=cypress/support/index.js:75:17)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  2                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     1 second                                                                         │
  │ Spec Ran:     visual.spec.js                                                                   │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

  (Screenshots)

  -  /home/pablo/projects/legion etrangere/projets/cra/cypress/screenshots/visual.spe     (1000x660)
     c.js/root.png                                                                                  
  -  /home/pablo/projects/legion etrangere/projets/cra/cypress/screenshots/visual.spe     (1280x720)
     c.js/test -- trying snapshots (failed).png                                                     

  (Video)

  -  Started processing:  Compressing to 32 CRF                                                     
  -  Finished processing: /home/pablo/projects/legion etrangere/projets/cra/cypress/v    (0 seconds)
                          ideos/visual.spec.js.mp4                                                  

====================================================================================================

  (Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  visual.spec.js                           00:01        1        -        1        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  1 of 1 failed (100%)                     00:01        1        -        1        -        -