Inconsistent screenshot resolutions

[smichaelsen]

We're building a Visual Regression Testing Suite based on webdriverio, wdio-image-comparison-service and Browserstack.

We have the problem that our tests fail regularly because the screenshots that are created vary in their resolution. We can reproduce the issue with browserstack and a local selenium grid; also with different browsers e.g. IE11 and Chrome.

Relevant versions

• node: v12.14.0
• npm: 6.13.4
• webdriverio: 5.22.3
• @wdio/sync: 5.20.1
• wdio-image-comparison-service: 1.12.2

Example test that shows that behaviour

const { expect } = require('chai');

describe('Page: News', () => {
    beforeEach(() => {
        browser.url('/my-test-page.html');
    });

    describe('Audio Player', () => {
        it('should look like the reference screenshot', () => {
            const progressbarAndTimerArea = { height: 40, width: 900, x: 232, y: 210 };

            const elem = $('.module-content-audio-player');
            elem.waitForDisplayed(3000);
            elem.scrollIntoView();
            elem.$('.module-content-audio-player__btn-play').click();
            browser.pause(4000);
            expect(
                browser.checkElement(elem, 'Audio Player Started', { blockOut: [progressbarAndTimerArea], })
            ).equal(0);
        });
    });
});

This test produces these two screenshots on the same page when run twice

We appreciate any hints how to track down the source of the problem.

[wswebcreation]

Hi @smichaelsen

This is kinda strange, this is a screenshot on the same OS and browser combination with the same resolution of the browser in the same session?

I'm pretty sure this is due to the environment / coding issue and not due to the module because the module only takes a screenshot of the current viewport. Can you share an instance of Browserstack which I can view to see what happens?

[smichaelsen]

We were able to fix this by putting browser.maximizeWindow(); into every test. Doesn't feel great, but it works.

[wswebcreation]

Hi @smichaelsen

Why not use something like this https://github.com/wswebcreation/wdio-image-comparison-service/blob/master/tests/configs/wdio.shared.conf.js#L43 Then your test will always have the same dimensions for every browser.

Going to close this issue because you found the root cause, but it's still open for discussion