search

FRSOURCE / cypress-plugin-visual-regression-diff

Perform visual regression test with a nice GUI as help. 💅 Only for Cypress!
MIT License
168 stars 22 forks source link

[Bug]: User should expect that the baseline images will get updated when the "Update screenshot" button is clicked #259

Open velosoLB opened 1 year ago

velosoLB commented 1 year ago

Describe the Bug

Discussed in https://github.com/FRSOURCE/cypress-plugin-visual-regression-diff/discussions/230

Given the following context:

On my side, I have set up a folder with some reference images (e.g. "reference_images/logo.png", "reference_images/progressBar.png", "reference_images/form.png"

When I run a test case, the screenshots are saved in a folder called "visual_testing". This folder is defined through the option "pluginVisualRegressionImagesPath" on Cypress config.

I then compare the images, one by one on my test script using cy.matchImage({ matchAgainstPath: 'reference_images/logo.png' }) and so on...

This works fine when just comparing the images, but when I want to update a reference image using the "Update screenshot" button, it doesn't work, because it updates the screenshots on the "visual_testing" folder instead of the ones on the "reference_images" folder.

Steps to reproduce

  1. Set up a folder with some reference images (e.g. "reference_images/logo.png", "reference_images/progressBar.png", "reference_images/form.png"
  2. Define a folder called "visual_testing" through the option "pluginVisualRegressionImagesPath" on Cypress config
  3. Call .matchImage({ matchAgainstPath: 'reference_images/logo.png' }) and the other reference images as well on your script
  4. Run the test case once using Cypress GUI
  5. Make some changes to the element you're testing, so that it has a visual difference higher than the threshold applied
  6. Run the test a second time. When the error appears, click on "See comparison" and click on the "Update screenshot" button
  7. The screenshot is saved in the "visual_testing" folder instead of updating the image in the "reference_images" folder

Expected behavior

The baseline images should be updated when the "Update screenshot" button is clicked

Additional context

Cypress v12.16.0

Package version

3.3.10

Device

PC

OS and version

Windows 10

Browser and version

Chrome, Edge

Node version

16.14.0

I can work on it by myself

Code of Conduct

mduivcap commented 1 year ago

We're experiencing the same issue with a similar structure (snapshot folder is not the default folder), the baseline image is not being updated when clicking the update screenshot button. The workaround is to manually delete this image and run the test again so it will be created again.