blockOut option is not working correctly

[amolinaalvarez]

Environment:

• Node.js version: 8.12.0
• NPM version: 6.4.1
• Browser name and version: Chrome 72.0.3626.96
• Platform name and version: Ubuntu 16.04
• Framework name and version: Protractor 5.4.2
• Plugin name and version: protractor-image-comparison 3.0.1
• webdriver-image-comparison version:0.3.0

Config of the automation framework + plugin here a basic example of how my conf.js looks like:

exports.config = {
   framework: 'jasmine',
   multiCapabilities: env.multiCapabilities,
   baseUrl: env.baseUrl,
   specs: [
     'image-comparator.spec.js',
   ],
   SELENIUM_PROMISE_MANAGER: false,
   directConnect: true,
}

Describe the bug I'm trying to use the method checkFullPageScreen with the option blockOut, bu it turns out that the comparator is not able to detect any change. In theory, there are many differences between the baseline and the actual version and I wanted to exclude one of them using the blockOut option.

However if I use the method checkFullPageScreen without the option blockOut, the comparator detects all the existed changes.

To Reproduce I used this code:

expect(await browser.imageComparison.checkFullPageScreen('profile', {
            blockOut: [{
              height: 48,
              width: 48,
              x: 1817,
              y: 16
            }],
})).toEqual(0);

it is not able to detect any change, even worse, this code neither:

expect(await browser.imageComparison.checkFullPageScreen('profile', {
            blockOut: [{
              height: 0,
              width: 0,
              x: 0,
              y: 0
            }],
})).toEqual(0);

However this one, yes:

expect(await browser.imageComparison.checkFullPageScreen('profile')).toEqual(0);

[wswebcreation]

Tnx for filing this bug. This is a bug from my side which I'm already fixing. I hope to release a new version tomorrow.

I'll keep you updated

[wswebcreation]

Hi @amolinaalvarez

I just released a new version called 0.3.1. Please remove your node_modules and your package-lock.json and try again. If it is fixed please close the issue

[amolinaalvarez]

@wswebcreation, thank you so much for your prompt reply! Right now, It is able to detect changes using the blockOut option. However, I do not know If it has anything to do with this, but when I use the blockOut option it still detects it as a difference, and, in theory, it should omit this region.

Am I missing something here? Thanks in advance.

[wswebcreation]

Can you please provide an example so I can debug it?

[amolinaalvarez]

My web application has a small avatar and this is the region that I want to omit, actually the rest of the content is the same. So I run these two examples:

1) Without using blockOut option:

browser.imageComparison.checkFullPageScreen('account-settings')).toEqual(0);

Expected 0.03 to equal 0.

As you can see it detects the avatar has changed:

2) Using blockOut option:

const menuAvatar = element(by.className('avatar'));
const location = await menuAvatar.getLocation();
const size = await menuAvatar.getSize();
expect(
    await browser.imageComparison.checkFullPageScreen('account-settings', {
        blockOut: [{
            height: size.height,
            width: size.width,
            x: location.x,
            y: location.y,
        }],
    }),
).toEqual(0);

Expected 0.03 to equal 0.

Unfortunately, the result is exactly the same, although in the diff image it seems that the avatar is excluded.

[wswebcreation]

You were right, there was an issue, thanks for pointing it out. I fixed it now with version 0.4.5 and I'll try to explain it below why you will still get a diff issue when you test it with the new version.

To get the latest version of the webdriver-image-compare-module you need to remove the node_modules and your package-lock.json and do a new npm install

What I did: I took a screenshot of this issue of your avatar, which will look like this, this is the baseline

I then replaced it with my avatar, which will look like this

I then compared the screenshots and I got this error Expected 0.01 to equal 0

Here you will see that there is a small line on the left and above my avatar that still fails, but if you look at the selected image in the dom below, you will also see that there is a small line that is not covered

Secondly you see an area on your image on the right of the avatar that fails. This has to do with the fact that you determine the position of the element before you do the screenshot. When the module takes the screenshot there are some things that er disabled by default, for example the scrollbars. This means that the image will get a different position during the screenshot than it had before the screenshot. You can prevent this by providing the option hideScrollBars: false.

I you still want the complete element to not be used in during the comparison, you should use the hideElements:[$(element(by.className('avatar')))] options, then the element will be hidden during comparison

[amolinaalvarez]

@wswebcreation thanks again for your update. For me is really important to keep the element in the image, so I'm using the hideScrollBars: false option, however there seems to be an issue using that option, let me show an example:

Baseline:

    const menuAvatar = element.all(by.className('avatar)).first();
    const location = await menuAvatar.getLocation();
    const size = await menuAvatar.getSize();
    expect(
      await browser.imageComparison.checkFullPageScreen('profile', {
        blockOut: [{
          height: size.height,
          width: size.width,
          x: location.x,
          y: location.y,
        }],
        hideScrollBars: false,
      }),
    ).toEqual(0);

Diff:

    const menuAvatar = element.all(by.className('avatar)).first();
    const location = await menuAvatar.getLocation();
    const size = await menuAvatar.getSize();
    expect(
      await browser.imageComparison.checkFullPageScreen('profile', {
        blockOut: [{
          height: size.height,
          width: size.width,
          x: location.x,
          y: location.y,
        }],
        hideScrollBars: false,
      }),
    ).toEqual(0);

As you can check it out, there is a shift in the blocked out region.

[wswebcreation]

@amolinaalvarez

Are you sure you are using the latest version of webdriver-image-comparison, please check your node_modules/webdriver-image-comparison and check the package.json and see which version you are using. As said, I've tested this with the above and it's now showing the diff as it should (you're still has the complete avatar as a purple diff)

[amolinaalvarez]

yes, I'm using the version 0.4.5 and I came across this issue.

[wswebcreation]

@amolinaalvarez Can you show an example project where I can check it? As said, I can't reproduce it, but it could be my problem

[amolinaalvarez]

Sorry for the delay, I just wanted to double check this issue on my project. Would it be useful to share the image in its three estates: baseline, actual and diff? Thanks again!

[wswebcreation]

Nope

It would be useful to have an example that shows the bug so I can verify and fix it. Based on my tests it works how it should be

[amolinaalvarez]

Once again sorry for the delay, It was impossible to work on this until a few days ago, I just checked it out again and everything seems to work. Thanks again @wswebcreation!