DevExpress / testcafe-browser-provider-saucelabs

This is the Sauce Labs browser provider plugin for TestCafe.
https://devexpress.github.io/testcafe/
MIT License
32 stars 41 forks source link

Migrate asset retrieval to platforms API #81

Closed alexplischke closed 1 year ago

alexplischke commented 1 year ago

Fixes https://github.com/DevExpress/testcafe-browser-provider-saucelabs/issues/80 by migrating the source of truth from outdated assets stored in the Sauce Labs wiki to the actual platforms API.

The code follows the API much closer in nature now, which brings us some breaking changes:

github-actions[bot] commented 1 year ago

Thank you for your contribution to TestCafe. We will review this PR.

apolonskiy commented 1 year ago

With new Saucelabs API there is a lost capability to provide [browserName@latest], where issue is in "latest" (it never comes in a list of available browser versions anymore).

Suggestion 1: Edit Readme to specify it's not possible to do that anymore. Suggestion 2: Add logic to check if passed browserVersion="latest" -> go to list of all browsers for current platform and browser and map it with largest number.

Just a thought, again, I've faced it while testing this PR locally and this will require us hardcode desired version, which is not the biggest deal but a breaking change it seems.

apolonskiy commented 1 year ago

Also, I'm not sure where this is coming from (maybe saucelabs-connector, so question not to this thread, but just heads up from Testcafe runner) - I get outdated version of macOS specified in console output when run TestCafe tests on anything newer than macOS (Mac) 10.15. It ALWAYS defaults to Mac 10.15 or Windows 10 accordingly.

saucelabs:chrome@111:Mac 12 Running tests in: Chrome 111.0.0.0 / macOS 10.15.7 (https://app.saucelabs.com/tests/ID) (bolded part is what comes as userAgent to reporter, which is not corresponding real browser / OS info)

Browser = saucelabs:chrome@111:Windows 11 Running tests in: Chrome 111.0.0.0 / Windows 10 (https://app.saucelabs.com/tests/ID) (bolded part is what comes as userAgent to reporter, which is not corresponding real browser / OS info)

alexplischke commented 1 year ago

With new Saucelabs API there is a lost capability to provide [browserName@latest], where issue is in "latest" (it never comes in a list of available browser versions anymore).

Suggestion 1: Edit Readme to specify it's not possible to do that anymore. Suggestion 2: Add logic to check if passed browserVersion="latest" -> go to list of all browsers for current platform and browser and map it with largest number.

Just a thought, again, I've faced it while testing this PR locally and this will require us hardcode desired version, which is not the biggest deal but a breaking change it seems.

I think I'll go with Suggestion 1, because the ability to run against the latest is not entirely lost. You can still do saucelabs:chrome:Mac 12 and Sauce Labs platform will pick the latest stable browser version.

apolonskiy commented 1 year ago

With new Saucelabs API there is a lost capability to provide [browserName@latest], where issue is in "latest" (it never comes in a list of available browser versions anymore). Suggestion 1: Edit Readme to specify it's not possible to do that anymore. Suggestion 2: Add logic to check if passed browserVersion="latest" -> go to list of all browsers for current platform and browser and map it with largest number. Just a thought, again, I've faced it while testing this PR locally and this will require us hardcode desired version, which is not the biggest deal but a breaking change it seems.

I think I'll go with Suggestion 1, because the ability to run against the latest is not entirely lost. You can still do saucelabs:chrome:Mac 12 and Sauce Labs platform will pick the latest stable browser version.

Oh well, I did not figure it out! This definitely makes sense then to just make sure everyone is aware of it :) Thank you for heads up! and for this MR overall!