Intern 3.4.4 and Leadfoot 1.7.2 breaks Firefox and IE on Saucelabs

[jonathanaustin]

We are currently using intern 3.4.4 but our tests for Firefox 45 on Linux (latest on any platform) and IE 11 (latest on Windows 10) started failing on Saucelabs.

As intern 3.4.0 was modified to use ~ for digdug and leadfoot version dependency, intern 3.4.4 is currently using leadfoot 1.7.2.

Leadfoot 1.7.2 appears to be making Firefox and IE fail due to it sending a GET window/rect command. The web drivers used by Saucelabs for Firefox and IE do not support this yet.

Forcing intern 3.4.4 to use leadfoot 1.7.1 allows Saucelabs to continue to work.

Found the following information helpful.

There is an issue in SeleniumHQ #3710 regarding the windows/rect command.

It appears the geckodriver required for Firefox that supports window/rect is version 0.16.0. The version of geckodriver currently used by Saucelabs is 0.9.0 for Selenium 2 and 0.11.1 for Selenium 3 (Saucelabs/selenium config details).

Saucelabs Log - IE

{
    "screenshot": null,
    "between_commands": 0.26799988746643066,
    "suggestion_values": [],
    "request": {},
    "HTTPStatus": 500,
    "result": {
      "hCode": 17659851,
      "stackTrace": [
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null
      ],
      "suppressed": [],
      "additionalInformation": "\nDriver info: driver.version: unknown",
      "localizedMessage": "GET /session/1635699d-d7fc-4346-9ebb-73cb928f57f5/window/rect\nBuild info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'\nSystem info: host: 'WIN-SB3ER6JQ6ME', ip: '172.20.32.24', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_91'\nDriver info: driver.version: unknown",
      "class": "org.openqa.selenium.UnsupportedCommandException",
      "buildInformation": null,
      "message": "GET /session/1635699d-d7fc-4346-9ebb-73cb928f57f5/window/rect\nBuild info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'\nSystem info: host: 'WIN-SB3ER6JQ6ME', ip: '172.20.32.24', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_91'\nDriver info: driver.version: unknown",
      "systemInformation": "System info: host: 'WIN-SB3ER6JQ6ME', ip: '172.20.32.24', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_91'",
      "cause": null,
      "supportUrl": null
    },
    "suggestion": null,
    "duration": 0.016000032424926758,
    "path": "window/rect",
    "method": "GET",
    "statusCode": 13,
    "timelineSeconds": 3.6510000228881836
  },

Saucelabs log - Firefox

{
    "screenshot": null,
    "between_commands": 0.4560840129852295,
    "suggestion_values": [],
    "request": {},
    "HTTPStatus": 500,
    "result": {
      "hCode": 529240,
      "stackTrace": [
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        null
      ],
      "suppressed": [],
      "additionalInformation": "\nDriver info: driver.version: unknown",
      "localizedMessage": "GET /session/1b86aab1-622d-4b84-8403-c248fb2819bb/window/rect\nBuild info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'\nSystem info: host: 'chef', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'i386', os.version: '3.13.0-83-generic', java.version: '1.8.0_74'\nDriver info: driver.version: unknown",
      "class": "org.openqa.selenium.UnsupportedCommandException",
      "buildInformation": null,
      "message": "GET /session/1b86aab1-622d-4b84-8403-c248fb2819bb/window/rect\nBuild info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'\nSystem info: host: 'chef', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'i386', os.version: '3.13.0-83-generic', java.version: '1.8.0_74'\nDriver info: driver.version: unknown",
      "systemInformation": "System info: host: 'chef', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'i386', os.version: '3.13.0-83-generic', java.version: '1.8.0_74'",
      "cause": null,
      "supportUrl": null
    },
    "suggestion": null,
    "duration": 0.018912076950073242,
    "path": "window/rect",
    "method": "GET",
    "statusCode": 13,
    "timelineSeconds": 3.1084039211273193
  },

[jason0x43]

Ironically enough, the window/rect check was added because calling window/$0/size in a Firefox 53 session will completely hang it (in Sauce or local Selenium). At least for local Selenium and BrowserStack, doing a window/rect test merely gets an unknown command error rather than causing a complete failure. In any case, though, both older and newer Firefox should work on Sauce; I'll look into it.

Note that Sauce Labs does, indeed, support window/rect when using a Firefox 53 session (which defaults to Selenium 3.4.0 and an unspecified version of geckodriver, apparently 0.16.0), and trying to use window/size instead of window/rect will break Firefox 53 sessions.

[jason0x43]

This should be fixed with Leadfoot 1.7.3 (at least, FF 33 and 55, and IE9-11, work properly on Sauce, BrowserStack, and TestingBot).