LironEr / cypress-mochawesome-reporter

Zero config Mochawesome reporter for Cypress with screenshots and videos
MIT License
159 stars 49 forks source link

`snapshots` and `__diff_output__` aren't copied to the report #54

Closed tfrijsewijk closed 2 years ago

tfrijsewijk commented 2 years ago

Environment

- OS: Windows 10
- Node: 16
- cypress-mochawesome-reporter: 2.3.0
- cypress: 9.1.1

What happened?

Nothing fancy: Followed your article https://liron-er.medium.com/cypress-html-reports-with-screenshots-made-easy-315a4c86c552, pretty basic stuff. I'm creating snapshots with

cy
        .visit(path)
        .matchImageSnapshot();

Config file

{
  "reporter": "cypress-mochawesome-reporter"
}

Relevant log output

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        9.1.1                                                                          │
  │ Browser:        Chrome 96 (headless)                                                           │
  │ Node Version:   v16.13.0 (C:\Program Files\nodejs\node.exe)                                    │
  │ Specs:          1 found (visual-regression.ts)                                                 │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

Remove output folder D:\git\the-company-directory\projecten\the-project-directory\src\cypress\reports\html

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  visual-regression.ts                                                            (1 of 1)
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db

Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating

  screenshots
    1) should match for https://website.project-test.nl/paginatypen/formulier/uitgebreid-formulier/
    √ should match for https://website.project-test.nl/paginatypen/tijdlijn/tijdlijn-opmaak-uitgebreid/

  1 passing (14s)
  1 failing

  1) screenshots
       should match for https://website.project-test.nl/paginatypen/formulier/uitgebreid-formulier/:
     Error: Image was 0.04794551138901295% different from saved snapshot with 2147 different pixels.
See diff for details: D:\git\the-company-directory\projecten\the-project-directory\src\cypress\snapshots\visual-regression.ts\__diff_output__\screenshots -- should match for httpswebsite.project-test.nlpaginatypenformulieruitgebreid-formulier.diff.png
      at Context.eval (https://website.project-test.nl/__cypress/tests?p=cypress\support\index.js:2055:17)

[mochawesome] Report JSON saved to D:\git\the-company-directory\projecten\the-project-directory\src\cypress\reports\html\.jsons\mochawesome.json

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        2                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  3                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     14 seconds                                                                       │
  │ Spec Ran:     visual-regression.ts                                                             │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

  (Screenshots)

  -  D:\git\the-company-directory\projecten\the-project-directory\src\cypress\snapshots\vis    (1000x4478)
     ual-regression.ts\__diff_output__\screenshots -- should match for httpswebsite.project-
     test.nlpaginatypenformulieruitgebreid-formulier.diff.png
  -  D:\git\the-company-directory\projecten\the-project-directory\src\cypress\screenshots\v     (1280x720)
     isual-regression.ts\screenshots -- should match for httpswebsite.project-test.nlpaginat
     ypenformulieruitgebreid-formulier (failed).png
  -  D:\git\the-company-directory\projecten\the-project-directory\src\cypress\snapshots\vis    (1000x1467)
     ual-regression.ts\screenshots -- should match for httpswebsite.project-test.nlpaginatyp
     entijdlijntijdlijn-opmaak-uitgebreid.snap.png

Read and merge jsons from "D:\git\the-company-directory\projecten\the-project-directory\src\cypress\reports\html\.jsons"
Copy screenshots folder from "D:\git\the-company-directory\projecten\the-project-directory\src\cypress\screenshots" to "D:\git\the-company-directory\projecten\the-project-directory\src\cypress\reports\html\screenshots"
Enhance report
Create HTML report
HTML report successfully created!
D:\git\the-company-directory\projecten\the-project-directory\src\cypress\reports\html\index.html

====================================================================================================

  (Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ×  visual-regression.ts                     00:14        2        1        1        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ×  1 of 1 failed (100%)                     00:14        2        1        1        -        -

PS D:\git\the-company-directory\projecten\the-project-directory\src>

Anything else?

image

The snapshots and diff's do exist:

/cypress/snapshots/visual-regression.ts/__diff_output__/screenshots -- should match for httpswebsite.project-test.nlpaginatypenformulieruitgebreid-formulier.diff.png
/cypress/snapshots/visual-regression.ts/screenshots -- should match for httpswebsite.project-test.nlpaginatypenformulieruitgebreid-formulier.snap.png
/cypress/snapshots/visual-regression.ts/screenshots -- should match for httpswebsite.project-test.nlpaginatypentijdlijntijdlijn-opmaak-uitgebreid.snap.png

I'm unable to determine whether this package or mochawesome-report-generator is responsible, but someone else opened an issue there: https://github.com/adamgruber/mochawesome-report-generator/issues/183

LironEr commented 2 years ago

If I understand correctly, you are using cypress-image-snapshot This reporter only copy screenshots from cypress screenshot path, and by default cypress-image-snapshot saves the screenshot to another directory

Can you try changing cypress-image-snapshot to save the images to the screenshot folder?

brnquester commented 2 years ago

@tfrijsewijk - FYI, I've posted my resolution: https://github.com/adamgruber/mochawesome-report-generator/issues/183#issuecomment-1009422056

Not sure if it will help you but there it is.

tfrijsewijk commented 2 years ago

@bruno-canada Thanks! I'll have a look and report back.