BitBar/BrowserStack: When("I $actionType baseline with name `$name` ignoring:$checkSettings") step take image from a browser haeader instead of page itself

[AntonD2203]

STR:

1. Run test.story on iOS BitBar (may be any available iOS device) device:

   Scenario: Take a baseline
   Given I am on a page with the URL 'https://github.com/'
   When I change context to element located `By.xpath(//header)`
   When I take screenshot
   When I COMPARE_AGAINST baseline with name `name` ignoring:
   |ACCEPTABLE_DIFF_PERCENTAGE |
   |15                                 |

2. Verify results for Checpoint image from When("I $actionType baseline with name $name ignoring:$checkSettings") step

Applied configuration:

web-application.sticky-header-size-percentage=40
selenium.grid.enabled=true
selenium.grid.url=https://eu-mobile-hub.bitbar.com/wd/hub
bdd.variables.global.platform=${selenium.grid.capabilities.platformName}
!-- Please set up your own API key in overriding properties so that you can access run details, video, etc
selenium.grid.capabilities.bitbar_apiKey=
!-- Capability prevents BitBar to use similar devices in case of unavailability of the selected device
selenium.grid.capabilities.bitbar_findDevice=true
!-- Capability that forces BitBar to remove files created during the testing (prevents session retention that could be visible to other users)
selenium.grid.capabilities.clearSystemFiles=True
selenium.grid.capabilities.platformName=iOS
selenium.grid.capabilities.deviceName=iPhone device
selenium.grid.capabilities.automationName=XCUITest
selenium.grid.capabilities.browserName=Safari
selenium.grid.capabilities.bitbar_device=Apple iPad Pro A1584 15.3

Expected result: Checpoint image is taken from part of the page which is in the context Actual result: Checpoint image is taken from part of the Safari browser header

NOTE: The same issue affects BrowserStack iOS devices: allure report - allure.zip session URL (yuri_cherevach@epam.com has access): https://automate.browserstack.com/dashboard/v2/builds/44a2686f272e989fac7e2e533797d8bba6b1cb1f/sessions/66a3ee581005aea29a9aca720d88d2f0cfca2ec6?auth_token=4b802276457b0d1d85def613488f09d7acbdb90fe74e565f59fc58554daf30c4

Device capabilities: Requested capabilities:

{
  "appiumVersion" : "1.19.1",
  "automationName" : "XCUITest",
  "bitbar_apiKey" : "IEtphJM8Vd88OdgaeyRVKvLgiE5GfVLE",
  "bitbar_device" : "Apple iPad Pro A1584 15.3",
  "bitbar_find
  "Device" : true,
  "bitbar_project" : "Saturn",
  "bitbar_testTimeout" : "1200",
  "browserName" : "Safari",
  "clearSystemFiles" : true,
  "deviceName" : "iPhone device",
  "deviceOrientation" : "portrait",
  "platformName" : "IOS",
  "screenshotWaitTimeout" : "20"
}

Session capabilities:

{
  "appium:networkConnectionEnabled" : false,
  "appium:showXcodeLog" : true,
  "appium:clearSystemFiles" : true,
  "appium:takesScreenshot" : true,
  "appium:appiumVersion" : "1.19.1",
  "appium:autoDismissAlerts" : true,
  "appium:automationName" : "XCUITest",
  "appium:locationContextEnabled" : false,
  "appium:screenshotWaitTimeout" : "20",
  "appium:databaseEnabled" : false,
  "appium:udid" : "00008110-001A69502E05801E",
  "appium:wdaLaunchTimeout" : 180000,
  "appium:xcodeConfigFile" : "/opt/testdroid/appium.xcconfig",
  "appium:deviceOrientation" : "portrait",
  "browserName" : "safari",
  "appium:webStorageEnabled" : false,
  "appium:deviceName" : "iPhone device",
  "appium:javascriptEnabled" : true,
  "platformName" : "IOS",
  "appium:usePrebuiltWDA" : true
}

Artifacts: allure report - allure.zip bitbar logs - bit_bar_logs.zip Recorded session - https://user-images.githubusercontent.com/66725458/218119833-5b775844-7feb-40bd-95bd-178e7ffd084b.mp4 Session URL (Anton_Drabkou1@epam.com has access) - https://cloud.bitbar.com/#testing/device-session/207165301/209105697/8851070

[AntonD2203]

Here is also results for scenario:

Scenario: Take a baseline
Given I am on a page with the URL 'https://github.com/'
When I COMPARE_AGAINST baseline with name `name` ignoring:
|ACCEPTABLE_DIFF_PERCENTAGE |
|15                         |

allure report - allure.zip session URL (yuri_cherevach@epam.com has access): https://automate.browserstack.com/dashboard/v2/builds/44a2686f272e989fac7e2e533797d8bba6b1cb1f/sessions/2f56f2b1e529a4df46aa6d45fadeb41a980a1756?auth_token=87c98d2c8aea43303db5d8921678a46298d9d38e1278808fb1f4879319381bbf

[valfirst]

@AntonD2203 is issue really reproduced on BrowserStack?

[valfirst]

@AntonD2203

"appiumVersion" : "1.19.1",

is there any reason to use so old Appium version?

[AntonD2203]

@AntonD2203

"appiumVersion" : "1.19.1",

is there any reason to use so old Appium version?

the version is determined by the bitbar and cannot be changed by us (the corresponding capability will be ignored on startup) In session logs version specified as [Appium] Welcome to Appium v1.22.2

[AntonD2203]

@AntonD2203 is issue really reproduced on BrowserStack?

yes, there is the report in NOTE section

[AntonD2203]

The bug is reproducible on v. 0.5.1

[valfirst]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

[AntonD2203]

Applied overriding properties: bdd.story-loader.batch-1.resource-location=story bdd.story-loader.batch-1.resource-include-patterns=test.story configuration.profiles=browserstack/ios/iPad9th/landscape,browserstack/web,web/tablet configuration.environments=whitesite/uat web-application.sticky-header-size-percentage=40 bdd.configuration.composite-paths=steps/*.steps bdd.configuration.formats=JSON known-issue-provider.fileName=known-issues.json known-issue-provider.detect-potentially-known-issues=false

Applied configuration.profiles properties: selenium.grid.enabled=true selenium.grid.username= selenium.grid.password= bdd.variables.global.platform=${selenium.grid.capabilities.platformName} selenium.grid.capabilities.platformName=iOS bdd.all-meta-filters=groovy: !skip && (${bdd.meta-filters}) selenium.grid.capabilities.browserName=Safari selenium.grid.capabilities.bstack\:options.os=iOS selenium.grid.capabilities.bstack\:options.osVersion=15 selenium.grid.capabilities.bstack\:options.realMobile=true selenium.grid.capabilities.bstack\:options.deviceName=iPad 9th selenium.grid.capabilities.bstack\:options.local=false selenium.grid.capabilities.bstack\:options.appiumVersion=1.22.0 bdd.variables.global.layout=tablet_large selenium.grid.capabilities.bstack\:options.deviceOrientation=landscape

Applied configuration.environments properties: template-processor.resolve-bdd-variables=true

[AntonD2203]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

Will check it

[AntonD2203]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

I've created a test repository to reproduce the bug. v0.5.4. is also affected - https://github.com/AntonD2203/bug_3629_vividus