browser.execute() does not accept functions

FuzzicalLogic commented 5 years ago

According to the WebDriver API and protocol.js, browser.execute() should support a function value. The following code does not work in chromedriver or geckodriver.

Bugs

As a function:

browser.execute(function() {
    return document.querySelector('p').innerText;
  },
  [],
  function(v) {
    console.log('execute()');
    console.log(v);
});

Work around It will accept a string as the function body only. Until this is fixed, one can use a javascript template value or a string.

browser.execute(`
    return document.querySelector('p').innerText;
  `,
  [],
  function(v) {
    console.log('execute()');
    console.log(v);
});

Stack

Windows 10
Nightwatch v1.0.18
chromedriver 2.45.0
geckodriver 1.14.1

Possibly related

#1920
#656

FuzzicalLogic commented 5 years ago

Justifications

As the other issues were closed without resolution, I think it is important to explain the reasoning behind re-opening these issues and putting them on a resolution track. The Webdriver API and nightwatch both provide a lot of resources for testing elements, interactions and resulting behaviors, but a significant portion of front-end code is not HTML/CSS. It is javascript. These often require testing in the front-end, and they do not always interact directly with elements. Of course, a great API should be simple, but some get quite large, and the tests would be difficult to maintain with all of that extra code.

Custom Elements (which are mostly standardly implemented) often have JS APIs in the form of methods, in addition to custom attributes and behaviors. Requiring js templates or strings to test such functionality, in the long run will result in more issues. The only other way to test these web components is with showroomjs, which I ultimately decided against in favor of Nightwatch because of its automation.

Errors from evaluated code are troublesome, at best. Some testing requires throws. If the test, itself, is flawed, this could result in false positives (or hella debugging) when test require a throw to pass. Of course, there are workarounds which could be documented, in the meantime. But, I do think that this needs to be on some resolution track, even if it is not priority.

FuzzicalLogic commented 5 years ago

After deeper review, my reference to #1920 is out of line. According to protocol.js#L1060:

/**
 * To ensure cross-browser compatibility, the specified function should not be in ES6 format 
 * (i.e. `() => {}`).
 */

beatfactor commented 5 years ago

Regular functions should work, as they are stringified, See https://github.com/nightwatchjs/nightwatch/blob/f794cdf63a9cf23e0c0ab9a390d6f6de9cf71a9b/lib/api/protocol.js#L1453.

If you're seeing a bug, could you also post a verbose log

FuzzicalLogic commented 5 years ago

@beatfactor, which log(s) would you prefer?

beatfactor commented 5 years ago

--verbose log output.

FuzzicalLogic commented 5 years ago

@beatfactor : To be more specific, do you want the debug.log, driver logs or selenium-server log? Or all of them? I ask, because all of them can get quite lengthy.

beatfactor commented 5 years ago

Just the log output from nightwatch when running with --verbose, not the selenium or driver logs.

beatfactor commented 5 years ago

FuzzicalLogic commented 5 years ago

@beatfactor Thank you for being patient with me. It's been some time since I've been using GitHub regularly and I forgot that CONTRIBUTING could be in .github/.

I am currently closing this issue, as I am unable to reproduce at this time. It most certainly was not working yesterday and the day before. I am relatively new to nightwatch, and my configuration has changed considerably since I initially posted the issue. So, it most certainly could have been due to misconfiguration. If I reproduce, I'll repost here and edit the title appropriately.

beatfactor commented 5 years ago

No problem. Feel free to re-open if you have a repro.

nightwatchjs / nightwatch