search

webdriverio / visual-testing

Image comparison / visual regression testing for WebdriverIO
https://webdriver.io/docs/visual-testing
MIT License
109 stars 41 forks source link

checkFullPageScreen() and checkElement() behavior is disturbed by elements with position:sticky #79

Closed DmytroCh closed 2 years ago

DmytroCh commented 2 years ago

Environment (please complete the following information):

Config of WebdriverIO + wdio-image-comparison-service

services: [
    ['chromedriver'],
    ['image-comparison', { // All options: https://github.com/wswebcreation/webdriver-image-comparison/blob/main/docs/OPTIONS.md
      baselineFolder:  join(process.cwd(), './test/sauceLabsBaseline/'),
      savePerInstance:  true, // Save the images per instance in a separate folder so for example all Chrome screenshots will be saved in a chrome folder like desktop_chrome
      autoSaveBaseline:  true, // If no baseline image is found during the comparison the image is automatically copied to the baseline folder when this is set to true
      blockOutStatusBar:  true, // Automatically blockout the status and address bar during comparions. This prevents failures on time, wifi or battery status. This is mobile only
      blockOutToolBar:  true, //Automatically blockout the tool bar. This is mobile only.
      fullPageScrollTimeout:  1500, // The timeout in milliseconds to wait after a scroll. This might help identifying pages with lazy loading.
    }]
],

Describe the bug I've noted that screenshots created by checkFullPageScreen() and checkElement() could be spoiled by element with position: sticky (see screenshots). In case of https://webdriver.io/ page top menu (<nav class="navbar navbar--fixed-top">) has this property and it jump on the screenshots. examplePaged-chrome-1920x1055-dpr-1 ytElement-chrome-1920x1055-dpr-1

To Reproduce Use next test cases and check screenshots in sauceLabsBaseline folder.

it('should compare successful with full page screen', async() => {
    await  browser.url('https://webdriver.io/');
    const  screenShot = await  browser.checkFullPageScreen('examplePaged');
    await  expect(screenShot).toEqual(0);
});

it('should compare successful specific DOM element', async() => {
    await  browser.url('https://webdriver.io/');
    const  el = await  $('.container .row iframe');
    const  screenShot = await  browser.checkElement(el, 'ytElement');
    await  expect(screenShot).toEqual(0);
});

Expected behavior

Additional context I was able to get proper screenshots by setting browser.debug() during tests execution and removing position property for <nav class="navbar navbar--fixed-top"> This issue could be related to:

wswebcreation commented 2 years ago

The method checkFullPageScreen can accept an options called hideAfterFirstScroll. This is an array of elements that will be hidden after the first automatic scroll of the method, see the full options list here

For the rest of the pages you need to fix this yourself because they don't automatically scroll.

Closing this because it's not an issue, but still open for conversation

DmytroCh commented 2 years ago

@wswebcreation thanks! Here is a working code (for interested):

it('should compare successful with full page screen', async() => {
        await browser.url('https://webdriver.io/');
        const navBar = await $('nav.navbar--fixed-top');
        const screenShot = await browser.checkFullPageScreen('examplePaged', { 
            hideAfterFirstScroll: [ 
                navBar
            ]
        });
        await expect(screenShot).toEqual(0);
    });

    it('should compare successful specific DOM element', async() => {
        await browser.url('https://webdriver.io/');
        const navBar = await $('nav.navbar--fixed-top');
        const el = await $('.container .row iframe');
        await el.scrollIntoView();
        const screenShot = await browser.checkElement(el, 'ytElement', { 
            hideElements: [ 
                navBar
            ]
        });
        await expect(screenShot).toEqual(0);
    });
wswebcreation commented 2 years ago

Thanks @DmytroCh for sharing the code!