Wrapper for nicer remote script execution

drslump commented 9 years ago

Motivation

We have found that often times our tests become very slow when basing them on the protractor/webdriver query API, specially when using iteration helpers on an ElementArrayFinder. To work around this we're using .executeScript to offload some of this work to the browser to avoid excessive roundtrips. It's a good pattern and works nicely except when the test fail with little information about what went wrong.

To solve that use case we've hacked a wrapper for .executeScript (and its async version) that offers a better experience. The code is available at https://gist.github.com/drslump/f71e9fd033ffd13a1123.

Example

With the above we can do stuff like this:

// Note that this function is marshalled to be executed in the browser so no closure available
var getAllTextsBySelector = remote(function getAllTextsBySelector (selector) {
  return Array.prototype.slice.call(document.querySelectorAll(selector))
  .map(function (node) {
    return node.innerText;
  });
});

getAllTextsBySelector('.info')
.then(function (texts) {
  expect(texts.length).toEqual(10);
  expect(texts[0]).toEqual('foo');
});

But the real benefit of the wrapper is to capture any errors thrown while executing the code on the browser and report it back in a nice way. For instance this is the error it produces for one of our tests (note how the stack trace maps to the original position in the code):

  1) A sample test:
     Unable to find any elements matching the selector ".people"
      at HTMLLinkElement.eval (/path/to/project/test/e2e/sample-e2e.js:40:27)
      at Function.n.extend.each (http://localhost:8085/scripts/libraries.js:2:2880)
      at n.fn.n.each (http://localhost:8085/scripts/libraries.js:2:847)
      at getAllTextsBySelector (/path/to/project/test/e2e/sample-e2e.js:39:44)
  ==== async task ====
  remote: getAllTextsBySelector(".people")
      at [object Object].webdriver.WebDriver.schedule (/path/to/project/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:345:15)
      at [object Object].to.(anonymous function) [as schedule] (/path/to/project/node_modules/protractor/lib/protractor.js:58:25)
      ...

Design

What the remote wrapper does is basically this:

Get the code to execute as a function (so we can get its name and the code to run)
Wrap the code with a try/catch so we can store any thrown error on window.$ERROR$
Wrap again inside an eval so we can include //# sourceURL=/path/to/file-calling-remote.js
Return a function the tests can use later on
When the function is called from the tests
- Use WebDriver's scheduler to send the execution command with a nice description
- When an error is reported:
- schedule a new script execution to fetch the window.$ERROR$
- modify the originally reported error with the information we got from the fetched one
- let the modified error reach the calling code
  Closing words

I'm happy to send a pull request for this but I lack knowledge about the internals of Protractor/WebDriver and would take me some time and guidance, so the question is, does Protractor benefit from having this new interface?

drslump commented 9 years ago

Forgot to mention that the current wrapper has been only fully tested on Chrome. The mapped stack traces should work on Firefox 36+ too but the wrapper probably will need some tuning.

juliemr commented 8 years ago

Thanks for this writeup, sorry for the lack of responses. I think we could definitely learn from the error reporting here.

One thing I am worried about is cross-browser compatibility - as soon as you step away from webdriver's native find methods, you've got to worry about that.

@cnishina can you look into whether we can add better debugging stuff to our own commands, and if it would be useful to add a command like @drslump 's to our core?

angular / protractor