client side browser validation

[bootstraponline]

It'd be great if sauce_ruby provided information about browser validation before attempting to run on Sauce with obviously invalid browsers. I see strange errors like:

    Failure/Error: Unable to find matching line from backtrace
     Selenium::WebDriver::Error::WebDriverError:
       unexpected response, code=403, content-type="text/plain"
       Invalid type received for 'version', got '9.0' of type '<type 'float'>' but Sauce OnDemand expects 'string'.

and

     Selenium::WebDriver::Error::WebDriverError:
       unexpected response, code=404, content-type="text/plain"
       Unsupported OS/browser/version/device combo: OS: 'Windows 2012 R2', Browser: 'ie', Version: '11.', Device: 'unspecified', Screen Resolution: '1280x1024'

[bootstraponline]

I think silently ignoring caps such as a default screen res on OS X 10.10 is fine. The completely invalid caps aren't good to ignore though.

[bootstraponline]

To expand on my comment a bit:

• If a capability isn't passing the whitelist, then an error should be raised instead of the current behavior of silently ignoring it. Problems with hashes are discussed in this article. Among the drawbacks is the lack of validation.
• If a valid white listed capability doesn't apply to a particular combination (screen res) then that's fine to ignore. The current behavior is to error even though that cap successfully applies to other combinations.

[DylanLacey]

I like the idea of complaining on non-white-listed capabilities. The problem with verifying capabilities is that, although there's a REST API, last I checked it wasn't super consumable; I'm hesitant to put in logic that relies on new versions of the gem being released to allow for new browsers, so I'd definitely want the REST API to be the thing driving this. That adds some spinup time to each test, as the API is queried (unless we do some sort of gross caching thing locally?).

[bootstraponline]

If it's not easy to get comprehensive validation info via REST then I'd be fine with a more targeted fix specifically for the screen res issue. There could be an override option which just ignores the logic if set.

[bootstraponline]

URL: http://saucelabs.com/rest/v1/info/platforms/:automation_api

That API will list all platforms. If we automatically generated classes based on the info then it'd enable people to be sure what they're requesting is valid (similar to watir autogenerating from html5 spec). Also if a new platform is added then they could just go back to the old string representation. I'll probably look into this since at work the goal is to enable people to easily write automation and specifying good combinations of platforms/browsers/capabilities is important.

[bootstraponline]

This is what I have in mind:

# returns ["Mac 10.6", "ipad", "1.0"]
puts Platform.mac_10_6.ipad.v1

# no method error because it's invalid
puts Platform.mac_10_6.ipad.v5

I'll try and finish up the code soon.

[bootstraponline]

@DylanLacey let me know what you think of the sauce_platforms gem.

[bootstraponline]

I solved the client side validation with sauce_platforms. The screen res example looks like this:

config['screen-resolution'] = '1280x1024'

config[:browsers] = [
  Platform.windows_8_1.firefox.v37,
  Platform.mac_10_10.safari.v8 + ['screen-resolution' => '1024x768'],
]