nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

Nightwatch.js does NOT support the W3C WebDriver API #1285

Closed blueimp closed 7 years ago

blueimp commented 7 years ago

nightwatchjs.org claims support for the W3C WebDriver API:

It uses the powerful W3C WebDriver API to perform commands and assertions on DOM elements.

However it does not implement this API, but rather implements the Selenium JSON Wire Protocol. e.g. it does not handle the W3C WebDriver spec for JSON deserialisation of an element.

Therefore it's currently not possible to use Nightwatch.js e.g. with the current Geckodriver without relying on an intermediate Selenium server. See also https://github.com/mozilla/geckodriver/issues/259

Is W3C Webdriver API support something that is planned? Maybe by relying on the official NodeJS selenium-webdriver library?

Thanks for your consideration!

beatfactor commented 7 years ago

Nightwatch does indeed not support the element response format of Geckodriver. However, this is the only driver which is not supported directly at this point. Chromedriver, Safari (10+) and Edge are all supported to various degrees and you can use them directly, without Selenium. The Selenium wire protocol has been made obsolete and will probably be replaced completely in the future by the W3C WebDriver API, which is why we have updated our references to point to the specification. There are still some incompatibilities and will take some time until Nightwatch is fully compatible with the W3C WebDriver API, and also browsers still have to implement the full WebDriver spec internally.

I would also like to refer to our contributing guidelines for submitting an issue. Supporting the Geckodriver directly at this point is not a priority for Nightwatch. I think if you take into account the existing backlog of bugs and feature requests, you will probably agree. Still, if you feel strongly that this should be implemented you are free to submit your feature request, but you should do so by following the aforementioned guidelines.

blueimp commented 7 years ago

Could you please explain which part of the guidelines I didn't follow? Cause I did read them in advance and am not sure what is wrong with this request.

Please do not misunderstand, I really appreciate this project and totally understand that this is not a priority. But given that Nightwatchjs.org does not follow the spec, I still think that the claim of using the W3C WebDriver API is misleading and therefore I do not understand why this issue was closed.

Chromedriver, which works very well with Nightwatch.js, implements the Selenium JSON Wire Protocol:

WebDriver is an open source tool for automated testing of webapps across many browsers. It provides capabilities for navigating to web pages, user input, JavaScript execution, and more. ChromeDriver is a standalone server which implements WebDriver's wire protocol for Chromium.

https://sites.google.com/a/chromium.org/chromedriver/

beatfactor commented 7 years ago

You are not reporting a bug or requesting a specific feature, but rather refer to something wrote on the website. So I'm not sure what you are requesting here.

blueimp commented 7 years ago

I expect the website and documentation to be correct about the features of this software. Therefore, if the website is correct, you can understand this as a bug report that the W3C Webdriver implementation is incorrect. It always fails when retrieving elements, as I described in my first post of this issue.

blueimp commented 7 years ago

Since this was written on the website, I also specifically asked:

Is W3C Webdriver API support something that is planned?

I do not blame you for not supporting it or for not prioritizing it, that's completely up to you. I was just surprised that it claims support for something that it doesn't and wanted to know if this is something which is planned.

beatfactor commented 7 years ago

We are not dealing with website issues here. There is a separate repo for that: https://github.com/nightwatchjs/nightwatch-docs.

Are you saying that because the Geckodriver response formats are incompatible, we are falsely claiming to support the W3C WebDriver API? You can still use the low level element commands with Geckodriver, you will need to write a bit more code, but it's possible. For example, the following works perfectly well:

module.exports = {
  'Demo test NightwatchJS.org' : function (client) {
    client
      .url('http://nightwatchjs.org')
      .element('css selector', 'body', function(res) {
        var element = res.value;
        client.elementIdDisplayed(element[Object.keys(element)[0]], function(result) {
          client.assert.ok(result.value, 'Element <body> is displayed');
        });
      })
      .end();
  }
};

So I'm not sure you're statement is accurate. Also, Safari and Edge work fine, for the most part.

blueimp commented 7 years ago

OK, I understand that it's possible to use the Webdriver Protocol commands when running against a W3C Webdriver implementation.

That means the Nightwatch.js Expect API, Assertions and Convenience Commands will never be available for implementations that follow the W3C Webdriver spec?

Maybe we have a different understanding what it means to support an API if only the low-level commands are supported.

Anyway, it's your decision, I simply wanted to know if this is ever planned and wanted to point out that the claim is misleading.

beatfactor commented 7 years ago

For someone who claims to appreciate Nightwatch you seem to be going to great lengths to try to disprove it.

That means the Nightwatch.js Expect API, Assertions and Convenience Commands will never be available for implementations that follow the W3C Webdriver spec?

It does not. They are available for implementations such as Safari and Edge already and will be made available for Geckodriver as well in the future, along with other implementations.

blueimp commented 7 years ago

Thanks, that was all I wanted to know. I'm sorry if this conversation came across as trying to disprove the quality or usefulness of this project, this was not my intention. I think a disagreement on technical details does not imply an intention of malice. I still disagree with you on the meaning of spec compliance, but I am content with your answer. I will also gladly continue to use and advertise this project despite this disagreement.