jaredpalmer / cypress-image-snapshot

Catch visual regressions in Cypress
MIT License
887 stars 160 forks source link

When using 'failOnSnapshotDiff=false' no snapshot-diffs are created #207

Open vglaeser opened 3 years ago

vglaeser commented 3 years ago

Hi,

I'm using the plugin with multiple e2e-tests. Actually, it is running quite well but when there are mutliple snapshots made in a single test only the first one is emitted since the test will fail afterwards. In order to avoid this, I tried failOnSnapshotDiff=false. With this option tests no longe fail, but diff-files are not created either. As far as I see, the variable pass (e.g. in plugin.js/matchImageSnapshotResult) seems to be set to true when using failOnSnapshotDiff=false even if there is a diff between existing and new snapshots. I would expect that diff_files are created anyway. Otherwise, you will propably miss other snapshot-diffs of the same test. (Assume a test that fails due to different snapshots. After accepting the difference you update existing snapshots which will overwrite any other potential differences)

Is this the intended behaviour or is there another way to create snapshot-diffs of all snapshots in a test?

Thanks in advance

francescq commented 2 years ago

This is the same behaviour I'm experiencing

Running with cypress openit respects failOnSnapshotDiff and the diff files are always created Running with cypress rundoes not create any diff files and looks like the snapshot filres are recreated with the previous file content

I would expect cypress run to always generate the diff files reglardless of failing or not the test on visual differences.

grahamemde commented 1 year ago

@jaredpalmer Is this behavior by design? I also have a use case for creating diff files when using cypress run and failOnSnapshotDiff=false. Would you be open to a change to make this more flexible?