Saucelabs integration is broken

[ricardorlg]

with the latest version, the Saucelabs integration become really complex, in the past, it was easy just to define saucelabs properties on serenity conf, but now with the use of sauce:options, is not possible to override properties on runtime, for example, things like this

 "sauce:options" {
          screenResolution = "1920x1440"
          build = "Caramel UI Test Automation - #{saucelabs.browserName} Desktop - test serenity upgrade"
          recordScreenshots = false
          extendedDebugging = "#{extended_debugging}"
        }

will not use the passed value for extended_debugging, also the AfterASauceLabsScenario class is not being called because the isActivated is still assuming the old version.

I propose to roll back the Saucelabs integration.

CC @wakaleo

[ricardorlg]

Also, the name of the test is not being used on saucelabs due to an error on BeforeASauceLabsScenario, the name capability should be passed in the sauce options not as a general one.

[alexanderkranga]

This broke for us as well, previously it used to get saucelabs properties from serenity.properties file and in our scripts we override the default values by passing sereniy.*=... properties through Gradle script. For example, having this in serenity.properties file would run tests remotely in SauceLabs and all browser related options would be passed to SauceLabs correctly:

saucelabs.recordScreenshots=false
saucelabs.timeout=360000
saucelabs.implicit.wait=10000
saucelabs.user.id=${userid}
saucelabs.access.key=${key}
saucelabs.url=http://${username}:${password}@ondemand.saucelabs.com:80/wd/hub

webdriver.driver=chrome
saucelabs.target.platform=Windows 10
saucelabs.driver.version=latest
saucelabs.screenResolution=1920x1080
saucelabs.browserVersion=latest

and to override these properties we would:

./gradlew test -Dwebdriver.driver=firefox -Dsaucelabs.target.platform=Safari ...

Now, starting from Serenity 3. this approach does not work and no `saucelabs.properties fromserenity.properties` file are took into account. Is there an up-to-date project that would show how to configure SauceLabs configuration correctly?

Thank you

[wakaleo]

In Serenity 3.*, you need to configure saucelabs in the serenity.conf file - see https://serenity-bdd.github.io/docs/cloud/saucelabs. You can override properties by using environment-specific configurations (see https://johnfergusonsmart.com/environment-specific-configuration-in-serenity-bdd/)

[alexanderkranga]

@wakaleo thank you for your response, we were able to fix our SauceLabs integration by using environments. Here is an example of serenity.conf file that we came up with:

# default environment
environment = local_chrome

environments {
    local_chrome {
        webdriver {
            driver = chrome
            capabilities {
                "goog:chromeOptions" = {
                    args = [ "--window-size=1920,1080", "--lang=en-us", "--disable-dev-shm-usage", "--homepage=about:blank", "--no-sandbox", "--ignore-certificate-errors" ]
                }
            }
        }
    }

    local_chrome_headless {
        webdriver {
            driver = chrome
            capabilities {
                "goog:chromeOptions" = {
                    args = [ "--window-size=1920,1080", "--lang=en-us", "--disable-dev-shm-usage", "--homepage=about:blank", "--no-sandbox", "--ignore-certificate-errors", "--headless" ]
                }
            }
        }
    }

    win10_desktop_chrome {
        sauce.active = true
        webdriver {
            driver = remote
            remote.url = "#{sauce.url}"
            capabilities {
                browserName = "chrome"
                browserVersion = "latest"
                platform = "Windows 10"
                "sauce:options" {
                    screenResolution = "1920x1080"
                    recordScreenshots = false
                    recordVideo = true
                    recordLogs = true
                    timeout = 360000
                    implicit.wait = 10000
                }
            }
        }
    }

    win10_desktop_firefox {
        sauce.active = true
        webdriver {
            driver = remote
            remote.url = "#{sauce.url}"
            capabilities {
                browserName = "firefox"
                browserVersion = "latest"
                platform = "Windows 10"
                "sauce:options" {
                    screenResolution = "1920x1080"
                    recordScreenshots = false
                    recordVideo = true
                    recordLogs = true
                    timeout = 360000
                    implicit.wait = 10000
                }
            }
        }
    }

    win10_desktop_edge {
        sauce.active = true
        webdriver {
            driver = remote
            remote.url = "#{sauce.url}"
            capabilities {
                browserName = "microsoftedge"
                browserVersion = "latest"
                platform = "Windows 10"
                "sauce:options" {
                    screenResolution = "1920x1080"
                    recordScreenshots = false
                    recordVideo = true
                    recordLogs = true
                    timeout = 360000
                    implicit.wait = 10000
                }
            }
        }
    }

    all {
        sauce.username = "..."
        sauce.key = "..."
        sauce.url = "https://ondemand.us-west-1.saucelabs.com/wd/hub"
    }
}

[alexanderkranga]

@wakaleo although there is another issue I found when trying to make google chrome run on a local machine in headless mode. According to example here https://github.com/serenity-bdd/serenity-junit-starter/blob/master/src/test/resources/serenity.conf#L10 I assume that I should be able to provide the headless.mode = true parameter to make chrome headless, but this never works. Also, chrome switches never work if provided by setting chrome.switches property as described here https://github.com/serenity-bdd/serenity-junit-starter/blob/master/src/test/resources/serenity.conf#L21.
The only way I was able to make it work in headless and read all additional capabilities is by using this in serenity.conf :

webdriver {
            driver = chrome
            capabilities {
                "goog:chromeOptions" = {
                    args = [ "--window-size=1920,1080", "--lang=en-us", "--disable-dev-shm-usage", "--homepage=about:blank", "--no-sandbox", "--ignore-certificate-errors", "--headless" ]
                }
            }
        }

I read through the code that initializes Google Chrome driver and I did not see any code that would read chrome.switches property. Please let me know if I am wrong. Thanks!