search

huridocs / uwazi

Uwazi is a web-based, open-source solution for building and sharing document collections
http://www.uwazi.io
MIT License
242 stars 80 forks source link

Cypress image snapshots not working #5834

Closed Zasa-san closed 1 year ago

Zasa-san commented 1 year ago

Describe the bug Cypress image snapshots are likely to be misconfigured since they are not properly reporting changes to the UI.

To Reproduce Steps to reproduce the behavior:

Expected behavior We should have a very low threshold of tolerance for visual changes in the UI and the test should fail.

Zasa-san commented 1 year ago

Doing something dramatic like adding another button does break the test. It could be that color changes are not enough to trigger an error.

Zasa-san commented 1 year ago

Right now https://github.com/meinaart/cypress-plugin-snapshots is an unmaintained repo. -> https://github.com/meinaart/cypress-plugin-snapshots/issues/220

This repo works, and mayor differences on snapshots will trigger an error. But we cannot keep using it if it's not maintained.

There are alternatives like https://github.com/simonsmith/cypress-image-snapshot
https://github.com/FRSOURCE/cypress-plugin-visual-regression-diff https://github.com/uktrade/cypress-image-diff

All of the above repos present problems with snapshot sizes in headless mode vs running the cypress app, or with different screen resolutions. You will get errors based on what environment you are running cypress on. If you create the snapshots while running the cy app the size of the screenshots will trigger an error when you check them in headless mode, and vice-versa.

For now I think we either have to a) Migrate to one of the above packages and then create the snapshots while running in headless mode (so that they pass in CI), and know that they will fail when using the cy app. b) Migrate and have two snapshot folders, one for headless and one for cy app. c) Keep using this repo until a better alternative appears.

@konzz @mfacar @grafitto

Zasa-san commented 1 year ago

For the time being we will continue using https://github.com/meinaart/cypress-plugin-snapshots since it works. If at a later time this package fails or presents problems we can migrate to another one, or come up with our own solution for visual testing.