jessezach
commented
3 years ago @josev814 Were you able to test this? Need to confirm that report generation does not break with these changes. Changes look good too except a minor comment.
Closed josev814 closed 3 years ago
jessezach
commented
3 years ago @josev814 Were you able to test this? Need to confirm that report generation does not break with these changes. Changes look good too except a minor comment.
josev814
commented
3 years ago Were you able to test this? Need to confirm that report generation does not break with these changes.
@jz-jess yes, I tested the report to ensure that it didn't break it. The report had it already baked in to check the existence of the actual and diff folders/files. If the folder/files don't exist, it doesn't include those segments in the file. The main chart still shows the correct pass/fail percentage chart. The tests themselves will show the baseline image being green. The depending on the cleanup option will show nothing in the rest of the columns, the actuals image and tolerance rating or the diff but not the actuals image and tolerance rating.
Example report when using all_passed as the cleanup parameter
Adding an option to remove actual and/or diff folders
The folders are only removed when a test is green and when open eyes has the parameter cleanup passed to it with one of the supported options documented in the README.md file
If an entry isn't entered or it doesn't exist in the supported list, no files will be cleaned up.
This is to help with reducing the amount of images and folders that are generated.
We use this library in Azure Pipelines and having the option to remove actuals and diffs of image comparisons that have passed is beneficial in reducing the output size. Additionally, when tests are passed, it's not as much of a concern as when they fail. So only including the actuals and diffs when a test fails helps to reduce the result size as well.