BackstopJS issue - same image with different screen size

[VinnyIla]

I am an Automation Engineer, using your BackstopJS for visual regression testing. I have been using this for a while and was quite impressed with its results. The scrubber functionality in the reporting makes it much useful to know where it failed comparatively.

Recently I started noticing that, the same browser size is launched differently, when TeamCity agent is set to auto login and start. Because of this, the test image is slightly different in dimensions to the reference image. I do set "RequireSameDimensions" to false. Is it possible for me to ignore screen size and just compare the content.

If you could provide me with some suggestions to resolve this, that would be much appreciated.

diff%: 4.18 diff-x: -636 diff-y: -257

Reference Image:

Test Image:

Diff Image:

[garris]

Thanks for posting. Have you tried scoping your screenshot to a particular dom element? This may help get around the window size issue.

[VinnyIla]

Yes, but i need to compare the screen as a whole. There are more than 50 screens like that we use for testing.

[garris]

I am not sure what is causing your screen size change but there are probably many many ways to address this. Maybe you could set an absolute width on your container when onReady runs? There is often no "right" way to address this kind of thing -- allow yourself to be creative. Good luck.

[rostislaw9]

Hello. Has this issue been diagnosed?

I've started using your BackstopJS recently and now am experiencing the same issue. I couldn't find the solution online so I tried to figure out what exactly causes the problem myself, but to no avail.

I'd like to note that it's not always that the same pair of images are of a different size, but sometimes they both are of a wrong size (which differs from those specified in the config). Thus, it could show successful tests results, but in fact target resolutions are not tested completely.

With this config:

"viewports": [
      {
        "label": "phone1",
        "width": 320,
        "height": 480
      }
    ]

Some results are ok: Some fail with different size:

[rostislaw9]

Worked this out by using "selectors": ["html"] to capture full page screenshots instead of using "selectors": ["document"]