Option to run all tests and get diff files at once even if there are failing ones

[pmoleri]

I want to run visual comparison for all my tests, but without stopping on failure, so I can compare all screenshots at once. This is convenient when upgrading a UI library that my project depends on.

I tried increasing the tolerance to 0.99 and it does the trick, but in this case I don't get the diff files which are very convenient.

Is there any way of do this? Can it be considered as a feature request?

[pmoleri]

I managed to do it by setting the RETRY_OPTIONS.doNotFail flag.

cypress-image-diff.config.ts:

module.exports = {
    FAILURE_THRESHOLD: 0.2,
    COMPARISON_OPTIONS: { threshold: 0.15 },
    RETRY_OPTIONS: {
        limit: 2, // max number of iterations
        delay: 200, // delay before the next iteration, ms
        log: true,
        doNotFail: true,  // <--- visual screenshot tests never fail
    },
};

cypress.config.ts:

import { defineConfig } from 'cypress';

export default defineConfig({
  ...
  e2e: {
    ...
    env: {
        cypressImageDiff: require('./cypress-image-diff.config'),
    },
    setupNodeEvents(on, config) {
        const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin');
        getCompareSnapshotsPlugin(on, config);
        ...
    },
  },
});

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.