search

codeceptjs / codeceptjs-resemblehelper

MIT License
27 stars 52 forks source link

Setting needsSameDimension flag to false isn't validating the difference of text between two locales. #114

Open sharath2106 opened 2 years ago

sharath2106 commented 2 years ago

We are using codeceptjs-resemblehelper to perform visual regression testing for our application that supports multiple locales. We are logging in with users mapped to these locales to achieve this.

Since our page is dynamic and it contains data. We are using I.seeVisualDiffForElement method and find the diff only for the elements that are static in the application. Some translations are rendering elements tad bigger than others.

On executing a negative case where the user belongs to the locale en-GB and compares the image for fr-FR, we see the below error - The base image is of 65 X 156 and actual image is of 62 X 156. Please use images of same dimensions so as to avoid any unexpected results.

searchBar-en-GB

searchBar-fr-FR

In order to ignore image size, we used the attribute needSameDimensions: false. But unfortunately, after using this attribute in our tests, the screen comparison is not working. The diff is not considered, and the tests are passing even though the locale is completely different.

Can you take a look at this issue?

puneet0191 commented 2 years ago

Hi @sharath2106 I will take a look at this issue, seems like a bug to me. thank you for reporting.

puneet0191 commented 2 years ago

Sure, I can't recall exactly which part causes this but feel free to send a patch for fixing this.

puneet0191 commented 2 years ago

@sharath2106 sorry I could not find enough time to verify, send PR, we will have it tested and make a new release.

sharath2106 commented 2 years ago

Hi @puneet0191 . I feel the issue is not because of the if statement. My fix which i was talking about was on this statement, but i think it's not. 

if (options.needsSameDimension && !data.isSameDimensions) {
            let dimensions1 = sizeOf(baseImage);
            let dimensions2 = sizeOf(actualImage);
            reject(new Error(`The base image is of ${dimensions1.height} X ${dimensions1.width} and actual image is of ${dimensions2.height} X ${dimensions2.width}. Please use images of same dimensions so as to avoid any unexpected results.`));
          }

Not sure how data.isSameDimensions is being fetched. Could you take a deeper dive on this?

sharath2106 commented 2 years ago

@puneet0191 Also, ignoredBox is not taking the precedence over sameDimensions -false flag. I think, this need to be looked and sorted out for our scripts to work.