saveAboveTolerance parameter not working

[pausk89]

Hi,

First of all I would like to thank you for so awsome job with this image comparison, it is really usefull for me. However, while I was trying different configuration parameters I noticed that saveAboveTolerance is not working for me. CheckElement method always creates an image in diff folder even is the difference percentage is lower than the one set in saveAboveTolerance parameter. Here is my configuration code:

const baselinePath = path.resolve(__dirname, `../../utils/images/baseline/`);
  const imagePath = path.resolve(__dirname, `../../utils/images/`);
  const protractorImageComparison = new ProtractorImageComparison({
    baselineFolder: baselinePath,
    screenshotPath: imagePath,
    autoSaveBaseline: true,
    debug: true,
    formatImageName: '{tag}',
    ignoreTransparentPixel: true,
    saveAboveTolerance: 10
  });
  protractorImageComparison.checkElement(applicationElement, name).then(function(res) {
    console.log(res);
  });

And the running output:

####################################################
compareOptions =  { isScreen: false,
  ignoreAntialiasing: false,
  ignoreColors: false,
  ignoreRectangles: [],
  ignoreTransparentPixel: true }
####################################################

6.32

[wswebcreation]

Hi @pausk89

That's strange, I just did a test and I see this (added some extra log to check this)

[14:46:42] I/hosted - Using the selenium server at http://localhost:4444/wd/hub/
options.saveAboveTolerance =  65
...saveAboveTolerance =  65
F.

Failures:

1) Scenario: Validate likes on name on top page # e2e/features/heroes.feature:4
   ✔ Given I open the heroes app # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✔ And "Wim Selles" has 9832 likes # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✔ When I like "Wim Selles" # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✖ Then the amount of likes of "Wim Selles" equals 9833 # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
       AssertionError
           + expected - actual

           -61.36
           +0

My config holds this

    /**
     * For protractor-image-comparison
     */
    browser.imageComparison = new protractorImageComparison({
      // Required
      baselineFolder: path.resolve(cwd, './e2e/baseline/'),
      screenshotPath: path.resolve(cwd, './.tmp/image-compare/'),
      // Optional
      autoSaveBaseline: false, // set this to true if you want to create an automatic baseline
      saveAboveTolerance: 65,
      debug: false            // set this to `true` to see some debugging and see blocked out images in the diff folder
    });

And I don't have an image in my diff-folder.

When I change my config to saveAboveTolerance: 60 then I see this in my log

[14:45:05] I/hosted - Using the selenium server at http://localhost:4444/wd/hub/
options.saveAboveTolerance =  60
...saveAboveTolerance =  60
F.

Failures:

1) Scenario: Validate likes on name on top page # e2e/features/heroes.feature:4
   ✔ Given I open the heroes app # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✔ And "Wim Selles" has 9832 likes # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✔ When I like "Wim Selles" # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
   ✖ Then the amount of likes of "Wim Selles" equals 9833 # Projects/rtc-demo-app/node_modules/cucumber/src/support_code_library_builder/define_helpers.js:90
       AssertionError
           + expected - actual

           -61.36
           +0

and an image in my diff folder

Can you check on which version you are on? This feature is available on version 1.7.0

[pausk89]

Hi @wswebcreation,

Thank you very much for your answer and all the details. I am using the version 1.7.0, after retesting, I noticed that this issue only happens to me when I have debug parameter set to true. When I set it to false the saveAboveTolerance is working fine. So, could you please check if it is working fine for you when you set debug: true?

Thank you in advance!

[wswebcreation]

It's in this line of code, https://github.com/wswebcreation/protractor-image-comparison/blob/master/index.js#L391

There's an OR, so if debug is true is well still save it. That's not a bug ☺️.

I'm going to close it for now, but it's still open for questions.

[pausk89]

Thank you very much for the clarification :)