webdriverio / webdriverio

Next-gen browser and mobile automation test framework for Node.js
http://webdriver.io
MIT License
9k stars 2.49k forks source link

[šŸ› Bug]: elementBlockOuts implementation missing from @wdio/visual-service #13255

Closed berekmerilorand closed 1 month ago

berekmerilorand commented 1 month ago

Have you read the Contributing Guidelines on issues?

WebdriverIO Version

8.39.1

Node.js Version

16.17.1

Mode

Standalone Mode

Which capabilities are you using?

local: {
        capabilities: [
            {
                'appium:maxInstances': '1',
                platformName: DEVICE_OS,
                'appium:deviceName': DEVICE_FULL_NAME || 'Android emulator',
                'appium:automationName': 'UiAutomator2',
                'appium:newCommandTimeout': Constants.WAIT_TIME_WORKERS_MS,
                'appium:udid': DEVICE_UUID || 'emulator-5554',
                'appium:hideKeyboard':
                    HIDE_KEYBOARD === undefined ? true : /^\s*(true|1|on)\s*$/i.test(HIDE_KEYBOARD),
                'appium:app': APP_PATH,
            },
        ],
    },

What happened?

I've tried to upgrade our current infrastructure which is using wdio-native-app-compare-service to visual-service, but unfortunately the elements are not hidden from the images/screenshots diffs.

Tried with both below flows:

                hideElements: [
                    await elem1,
                    await elem2,
                ],
                elementBlockOuts: [
                    {
                        element: await elem1,
                    },
                    {
                        element: await elem2,
                    },

Infrastructure:

App: Native App (RN based)
Node: 16.17.1
WDIO: 8.39.1
Appium: 2.11.2
visual-service: 5.2.0

wdio.base.visual.config.ts file contents:

export const visualDiffConfig = {
    misMatchPercentage: 0.4,
    largeMisMatchPercentage: 0.8,
    specFileRetries: process.env.CIRCLECI ? 1 : 0,

    services: [
        'visual',
        {
            baselineFolder: `./e2e/screenshots/${process.env.DEVICE_OS}/expected`,
            formatImageName: `{tag}-{deviceName}`,
            screenshotPath: `./e2e/screenshots/${process.env.DEVICE_OS}`,
            savePerInstance: true,
            autoSaveBaseline: true,
            blockOutToolBar: true,
            blockOutStatusBar: true,
            ignoreAlpha: true,
            scaleImagesToSameSize: true,
        },
    ],
};

tsconfig.json file contents:

        "types": [
            "node",
            "jest",
            "@wdio/globals/types",
            "@wdio/mocha-framework",
            **"@wdio/visual-service",**
            "expect-webdriverio/jest",
        ],

What is your expected behavior?

Elements blocks to be hidden from diff images/screenshots.

How to reproduce the bug.

  1. Run a visual test on a native app with latest visual-service version and make sure baselines (images/ss are saved).
  2. Run the test again with elements to be hidden mentioned in blockOuts list (you've to make sure that there is indeed a difference for that element, in order for that part of the screen to be marked with green colour in diffs folder).

Relevant log output

Diff is not detected (element is not hidden from the diff picture).

Code of Conduct

Is there an existing issue for this?

wswebcreation commented 1 month ago

Closing this one because it's not part of WDIO, but is it's own project https://github.com/webdriverio/visual-testing/issues/447