codeceptjs / codeceptjs-resemblehelper

MIT License
27 stars 53 forks source link

Optionally skip "output" & "diff" upload to S3 #139

Open mbrodala opened 1 month ago

mbrodala commented 1 month ago

If S3 (AWS, DigitalOcean, etc.) is configured, this helper downloads and uploads output (actual), base and diff images to the S3 storage.

However for a regular CI workflow it does not make sense to have diff images uploaded:

  1. They only make sense for the particular CI run with the particular source code state
  2. The result in the S3 storage is undefined in case of other CI runs executed in parallel or other CI pipelines

Thus it would be useful to have an option to skip the upload of diff images to S3. Instead they should only be stored locally in this case.

Notice that the same goes for output (actual) for the same reasons listed above. Updating the base images should always be done locally

mbrodala commented 3 weeks ago

We have worked around these issues ATM:

  1. Have a base version as configuration which will be used as prefix for all screenshots (e.g. v2-my-screenshot.png)
  2. Have an additional prefix as configuration, based on the Git commit ref name in GitLab ($CI_COMMIT_REF_SLUG) and the current date locally (e.g. foo-feature-v2-my-screenshot.png, bar-feature-v2-my-screenshot.png) and use this for I.saveScreenshot() and I.seeVisualDiff()
  3. Use compareWithImage on I.seeVisualDiff() to use the unprefixed base screenshot (v2-my-screenshot.png)

This way there are no conflicts anymore on S3. The base folder contains the versioned but unprefixed base screenshots and diff / output contain the versioned and prefixed diff/actual screenshots. The base screenshots are basically shared between all Git branches but said branches have their specific output and diff screenshots.

Still an option to skip the (still pointless) upload of output and diff would be useful.