Support latest w3c WebDriver spec changes

[vladikoff]

This is a tracking issue to fix compatibility issues between the WebDriver Spec and Selenium. The data structures are currently not compatible.

Ref https://github.com/jgraham/wires/issues/45#issuecomment-175276931

cc @automatedtester @jason0x43

[AutomatedTester]

I will start working on a patch as soon as possible. The idea is that we will be compliant in both and just sniff to know what to do.

[vladikoff]

@jason0x43 we need to triage this. I'm guessing once Firefox 48 is released on August 2nd, 2016 Intern won't work with latest Firefox anymore...

[vladikoff]

Maybe it also makes sense to move to use https://www.npmjs.com/package/selenium-webdriver because that's the thing that keeps up to date with the spec.

[dylans]

@vladikoff thanks for the warning. Is selenium-webdriver their alternative to leadfoot? With a very cursory glance, their usage examples look fairly ugly from the perspective of how a JS dev would write an API.

[jason0x43]

Intern already has issues with FF. FF 47 doesn't work with FirefoxDriver, and GeckoDriver breaks when accessing window vars, so we're kind of stuck for the moment (at least for the current stable FF build).

I implemented a couple of checks for specific WebDriver features in Intern recently, and several of the existing checks for broken JsonWire features also work for WebDriver (which doesn't have some JsonWire equivalents). I haven't pursued this much, though, since there aren't any widely deployed W3C compliant drivers (that I'm aware of). Time to look into this more...

[vladikoff]

Intern already has issues with FF. FF 47 doesn't work with FirefoxDriver,

That was a different issue with certificates that is going to be fixed in 47.0.1 Ref: https://github.com/seleniumhq/selenium/issues/2110

GeckoDriver breaks when accessing window vars

That was uplifted to latest FF 48 beta (@andreastt correct me if I'm wrong)

[andreastt]

GeckoDriver breaks when accessing window vars

@vladikoff @jason0x43 This was fixed in bug 1123506 which indeed has been uplifted to Firefox 48.

[vladikoff]

I'm getting issues with an incompatible add-on with Firefox 47.0.1 and have to set the following to enable the WebDriver add-on to work:

  setPreference('extensions.checkCompatibility.47.0', false);

In terms of updating leadfoot we should probably aim for Firefox 48 compat and skip 47. (47.0.1 support might be fixed by adding the pref)

[vladikoff]

Firefox 48 stable is out. To run tests we now need to download the geckodriver and set something like this:

 profile.setPreference('marionette', true);
 profile.setPreference('webdriver.gecko.driver', '/stuff/geckodriver');

I'm getting multiple errors accessing page properties. however session creation works.

[andreastt]

The WebDriver specification mandates that conformant implementations should use strings to identify errors instead of numbers. As can be seen in https://github.com/mozilla/geckodriver/issues/176, the script timeout error that is returned from Marionette is not mapped down to the ScriptTimeoutException in the Leadfoot binding.

The error code lookup you have in lib/statusCodes.js for Selenium additionally needs a mapping for WebDriver-conformant remote ends, such as javascript/node/selenium-webdriver/lib/error.js in Selenium.

[meteorsnows]

The CodeceptJS having helpers libraries for webdriverio, Protractor, Selenium WebDriver JS, NightmareJS. Does Intern should having this feature, or should it abstract via leadfoot? But To be honest, there is too many library there, compatibility issue would be broken for certain library at certain point.

CodeceptJS has it own merit for support API that the developer comfortable to. I suggest leadfoot at least still maintain API compatibly for WD.js. Or Maybe building abstraction API layer on top of Selenium WebDriver JS which Protractor also depend on.

Furthermore, official Selenium WebDriver JS seem to be the only one able to do following, no sure it entirely working or not

1: +==== Controlling the Host Browser ====

2: There are a few caveats to using WebDriverJS in the browser

Reference https://groups.google.com/forum/#!topic/selenium-developer-activity/Yssd-rQzIUE

[dylans]

The CodeceptJS having helpers libraries for webdriverio, Protractor, Selenium WebDriver JS, NightmareJS.

CodeceptJS certainly interesting. In many ways its focus is complementary to Intern (where the focus is primarily browser apps, whereas CodeceptJS is mostly interested in Node.js tests), which leads to some very different design decisions (Intern is all async, CodeceptJS is all sync, etc.).

Does Intern should having this feature, or should it abstract via leadfoot? But To be honest, there is too many library there, compatibility issue would be broken for certain library at certain point.

It's not entirely clear to me where the right abstraction is as I'd need to spend more time understanding CodeceptJS first. That said, I would think that tests would prevent compatibility regressions. If only we knew of a few good test libraries. :)

CodeceptJS has it own merit for support API that the developer comfortable to. I suggest leadfoot at least still maintain API compatibly for WD.js. Or Maybe building abstraction API layer on top of Selenium WebDriver JS which Protractor also depend on.

WD.js was used as part of Intern 1 prior to Leadfoot. Some of our motivation is summarized in https://www.sitepen.com/blog/2014/05/07/whats-next-for-intern/ . Basically at the time WD.js was not keeping pace with the spec, and had a lot of bugs, and no ability to feature test for web driver features that were broken in various driver instances.

Furthermore, official Selenium WebDriver JS seem to be the only one able to do following, no sure it entirely working or not

The Selenium WebDriver JS was then introduced after Leadfoot, as the spec moved from Selenium to the W3C. The challenge we have today is a moving spec, and two versions of it that are supported in different drivers, and not with full consistency.

+==== Controlling the Host Browser ==== There are a few caveats to using WebDriverJS in the browser Reference https://groups.google.com/forum/#!topic/selenium-developer-activity/Yssd-rQzIUE

Leadfoot was definitely designed as something to run from Node.js, over the WebDriver Wire Protocol, to a WebDriver that controls a browser. I don't believe we've ever considered running it in a browser, so I'm not sure what you would lose, but I imagine if it was made to work, it would have similar limitations to the caveats mentioned for Selenium WebDriverJS. I've not spent time (yet) to think through what the benefits would be (the link above clearly talks about them, but I've not read it in detail at this time).

[vladikoff]

this is not needed anymore. See this comment on how to configure Intern with latest geckodriver: https://github.com/theintern/digdug/issues/30#issuecomment-238050664