Config "FAILURE_THRESHOLD" did not work

[Strezzo06]

Now i use version 2.1.3 Cypress v13.0.0

FAILURE_THRESHOLD: 0.1

Result

I try to add other config

It's working

[tamasmagyar]

I was not able to repro with:

• Cypress 13.1.0 or above (unfortunately I had issues with 13.0.0)
• cypress-image-diff-js 2.1.3

Config file (cypress-image-diff.config.cjs):

const config = {
  ROOT_DIR: '',
  FAILURE_THRESHOLD: 0.07,
  COMPARISON_OPTIONS: { threshold: 0.2 },
  RETRY_OPTIONS: {
    log: true,
    limit: 3,
    timeout: 1000,
    delay: 1000
  },
  JSON_REPORT: {
    OVERWRITE: true,
  }
};
module.exports = config;

Result:

---

@Strezzo06 Can you share the smallest amount of code that reproduces the bug? This helps in isolating the specific problem without the complexity of a larger codebase.

I created an initial project which you can configure to repro the bug, thank you in advance for the help! Playground repository: https://github.com/tamasmagyar/cypress-image-diff-repro

[Strezzo06]

My config file

// cypress-image-diff.config.cjs
const config = {
    ROOT_DIR: '',
    FAILURE_THRESHOLD: 0.1,
    COMPARISON_OPTIONS: { threshold: 0.2 }, // default to { threshold: 0.1 }
    RETRY_OPTIONS: {
        log: true,
        limit: 3, // max number of iterations
        timeout: 15000, // time limit in ms
        delay: 1000, // delay before next iteration, ms
    },
    JSON_REPORT: {
        // FILENAME: 'cypress_visual_report', // default "report_[datetime].json"
        OVERWRITE: true, // default true
    },
};
module.exports = config; 

cypress.config.js

const { defineConfig } = require('cypress');
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/plugin');

module.exports = defineConfig({
    defaultCommandTimeout: 15000,
    viewportWidth: 1440,
    viewportHeight: 900,
    waitForAnimations: true,
    scrollBehavior: 'center',
    e2e: {
        setupNodeEvents(on, config) {
            // // import and initialise the cypress image diff plugin
            getCompareSnapshotsPlugin(on, config);

            //Separate CV/LCV
            if (config.env.WEB === 'lcv') {
                return {
                    specPattern: ['cypress/e2e/lcv/**/product*.cy.js'],
                };
            } else if (config.env.WEB === 'cv') {
                return {
                    specPattern: ['cypress/e2e/cv/**/*.cy.js'],
                };
            }
            return getCompareSnapshotsPlugin(on, config);
        },
        baseUrl: 'http://localhost:3000',
        specPattern: 'cypress/e2e/**/*.{js,jsx,ts,tsx}',
    },
});

I will try to run on cypress 13.6.1 to compare with cypress 13.0.0

[Strezzo06]

I found the root cause. My problem is this condition that i added before return config

//Separate CV/LCV
            if (config.env.WEB === 'lcv') {
                return {
                    specPattern: ['cypress/e2e/lcv/**/product*.cy.js'],
                };
            } else if (config.env.WEB === 'cv') {
                return {
                    specPattern: ['cypress/e2e/cv/**/*.cy.js'],
                };
            }

When i remove that condition. My config can work properly.

[tamasmagyar]

Can you try the following changes, please?

I think assigning to config.specPattern should do job, you don't have to return with that.

const { defineConfig } = require('cypress');
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/plugin');

module.exports = defineConfig({
    defaultCommandTimeout: 15000,
    viewportWidth: 1440,
    viewportHeight: 900,
    waitForAnimations: true,
    scrollBehavior: 'center',
    e2e: {
        setupNodeEvents(on, config) {
            //Separate CV/LCV
            if (config.env.WEB === 'lcv') {
                                config.specPattern = 'cypress/e2e/lcv/**/product*.cy.js'
            } else if (config.env.WEB === 'cv') {
                config.specPattern = 'cypress/e2e/cv/**/*.cy.js'
            }
            return getCompareSnapshotsPlugin(on, config);
        },
        baseUrl: 'http://localhost:3000',
        specPattern: 'cypress/e2e/**/*.{js,jsx,ts,tsx}',
    },
});

[Strezzo06]

Can you try the following changes, please?

I think assigning to config.specPattern should do job, you don't have to return with that.

const { defineConfig } = require('cypress');
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/plugin');

module.exports = defineConfig({
  defaultCommandTimeout: 15000,
  viewportWidth: 1440,
  viewportHeight: 900,
  waitForAnimations: true,
  scrollBehavior: 'center',
  e2e: {
      setupNodeEvents(on, config) {
          //Separate CV/LCV
          if (config.env.WEB === 'lcv') {
                                config.specPattern = 'cypress/e2e/lcv/**/product*.cy.js'
          } else if (config.env.WEB === 'cv') {
              config.specPattern = 'cypress/e2e/cv/**/*.cy.js'
          }
          return getCompareSnapshotsPlugin(on, config);
      },
      baseUrl: 'http://localhost:3000',
      specPattern: 'cypress/e2e/**/*.{js,jsx,ts,tsx}',
  },
});

Thank you so much, it's work.