stof commented 6 years ago

Support for the JsonWire selenium protocol is fading out in favor of the W3C spec:

Firefox's geckodriver implements only the W3C protocol
Safari's driver uses the W3C driver by default (see https://github.com/minkphp/Mink/issues/764)
Chrome's driver is working on supporting the W3C protocol (they still implement JsonWire for now)
Edge's driver uses the W3C driver by default in its next release (so same situation than Safari will happen soon: https://docs.microsoft.com/en-us/microsoft-edge/webdriver#webdriver-server-command-line-flags)

We will have some work to do to make this happen.

The instaclick/php-webdriver library that we are using has a beta version of 2.0 which adds support for interacting with a W3C server. But they won't provide an abstraction layer to make the same code work for both variants (see https://github.com/instaclick/php-webdriver/issues/84). It will be up to our code to choose the right way (and we will need https://github.com/instaclick/php-webdriver/issues/86 to be able to know which one we need). Due to the fact that we expose the webdriver session to userland, we would have to consider this upgrade as a BC break.

Another option could be to migrate to facebook/webdriver, which seems a bit more popular this days (https://packagist.org/?query=webdriver). It is currently adding support for the W3C spec (WIP is at https://github.com/facebook/php-webdriver/pull/560), and does so by abstracting the differences internally). Doing such migration would also require making it a major release, as we would change the API we expose when exposing the session.

stof commented 6 years ago

https://github.com/minkphp/MinkSelenium2Driver/issues/288 was created to refer to the first solution for this topic (but does not describe the implication, and does not seem to care about the W3C support in the PR implementing it)

stof commented 6 years ago

287 should be done first before this work (we should switch to Chrome on Travis, as there is no way to run recent Firefox versions with the JsonWire protocol).

stof commented 6 years ago

The more I look at instaclick/php-webdriver 2 "supporting" the WEC protocol (currently in RC), the more I think we should rather migrate to the facebook package (for which the PR supporting the W3C protocol inside their abstraction is almost ready AFAICT).

stof commented 6 years ago

Btw, if we look at https://packagist.org/packages/instaclick/php-webdriver/dependents and the download counts on https://packagist.org/?query=webdriver, I think most of the downloads of instaclick/php-webdriver are coming from Mink using it (our driver has 11 176 049 downloads, the library has 11 517 849, so this only means 340k downloads not coming from us, which is 3%). Among the 12 packages depending on instaclick/php-webdriver on Packagist, here is my analysis:

1 is us
2 are (abandoned) forks of us, (one direct fork, and a ghost-driver which duplicated our codebase instead of depending on us)
3 are about locking exact versions of deps in the Drupal ecosystem and so are related to us (as Drupal is using Mink)
1 is an abandoned experiment from the Laravel ecosystem, but laravel/dusk has been based on the facebook library instead.
1 is a fork of the previous experiment
the remaining 4 look abandoned (last commits either in 2016 or 2014)

So we are the only ones keeping instaclick/php-webdriver alive in the open.

aik099 commented 6 years ago

https://github.com/minkphp/MinkSelenium2Driver/issues/293#issuecomment-426670405

Unfortunately Facebook Web Driver is now requiring PHP 7 and that was main reason why PR still isn't merged.

stof commented 6 years ago

@aik099 as we need to make a new major release of the driver anyway, I'm fine with requiring PHP 7.1+ for it. All older PHP versions will be EOLed by the end of December.

People really wanting to use PHP 5.x could still use Selenium2Driver 1.x and use Chrome (or another browser with a driver supporting the legacy protocol)

aik099 commented 6 years ago

Agreed. We'll then have new driver require PHP 7.1, but Mink itself stays with current 5.x requirement.

stof commented 6 years ago

Well, Mink itself will probably bump its own requirement in the future too, but I don't have a roadmap for that (as long as we don't make changes benefiting from it, there is no strong need)

nirbhaypatil commented 6 years ago

just wanted to know how to use instaclick/php-webdriver:2.0.0-RC2 with mink and mink extension ? dependencies mismatch error comes up.

{ "require": { "behat/behat":"~3", "behat/mink": "1.7.1", "behat/mink-extension": "2.2", "behat/mink-selenium2-driver": "1.3.1", "behat/mink-goutte-driver": "1.2.1", "drupal/drupal-extension":"3.1.5", "sensiolabs/behat-page-object-extension": "dev-master" }, "config": { "bin-dir": "bin/" } }

Your requirements could not be resolved to an installable set of packages.

stof commented 6 years ago

currently, you cannot do it directly.

nirbhaypatil commented 6 years ago

even with facebook/webdriver + minkextension? can you suggest other ways to do .

stof commented 6 years ago

I don't understand your question. You asked whether you can use instaclick/php-webdriver:2.0.0-RC2, and now you say even with facebook/webdriver ?. I don't see how they are related.

nirbhaypatil commented 6 years ago

Sorry about confusion ,my question is Can we use 1) mink + minkextension + instaclick/php-webdriver:2.0.0-RC2 OR 2) mink + minkextension + facebook/webdriver to execute tests on firefox > 48

stof commented 6 years ago

none of that is possible today, which is precisely what this issue is about:

using facebook/webdriver would involve rewriting the driver entirely, as it is a different library. You cannot just install it and expect it to be used
installing instaclick/php-webdriver:2.0.0-RC2 will not make it work with recent Firefox, even if you trick composer into accepting it. instaclick/php-webdriver does not abstract the differences between the W3C and JsonWire protocol, but exposes APIs supporting both and expects the calling code to use the right one, which MinkExtension does not do it yet. And we cannot do it based on their RC2 releases:
- they don't expose whether the session is a W3C or JsonWire one, so we would have to guess it currently (without having the info to detect it reliably): https://github.com/instaclick/php-webdriver/issues/86
- their timeout handling (one of the few APIs they actually abstract) is broken with W3C: https://github.com/instaclick/php-webdriver/issues/88
- the error handling is broken with W3C: https://github.com/instaclick/php-webdriver/issues/87

So we are still discussing in which direction we want to rewrite the driver (supporting the W3C spec on top of instaclick/php-webdriver:2 also involves lots of rewriting in the driver).

oleg-andreyev commented 5 years ago

@stof recently we've been trying to migrate this library to facebook/php-webdriver, mostly because of syn.js which bundled into this library and because instaclick/php-webdriver is not supported that actively.

So far my results: Diff: https://github.com/minkphp/MinkSelenium2Driver/compare/master...oleg-andreyev:master Travis Build (with Chrome): https://travis-ci.org/oleg-andreyev/MinkSelenium2Driver/jobs/462514905 Travis Build (with FF): https://travis-ci.org/oleg-andreyev/MinkSelenium2Driver/jobs/462514904 Travis: https://travis-ci.org/oleg-andreyev/MinkSelenium2Driver/builds/462514900

There are few issues with Chrome eg: Keyboard event, but it it's related to assert of driver-testsuite. Also there are issues with facebook/webdriver and with FF

phil-davis commented 5 years ago

Note: chrome+chromedriver 75 has recently come out and now defaults to w3c So now I think MinkSelenium2Driver users need to set chromeOption w3c to false

Also see https://github.com/SeleniumHQ/docker-selenium/issues/922 where I am having this trouble with the latest https://github.com/SeleniumHQ/docker-selenium 3.141.59-palladium which has chrome 75.

AlexSkrypnyk commented 5 years ago

@phil-davis Looks like we were debugging this at the same time :))

This is the full fix in behat.yml:

  extensions:
    Behat\MinkExtension:
      selenium2:
         capabilities: { "browser": "chrome", "version": "*", "marionette": true, "extra_capabilities": { "chromeOptions": { "w3c": false } } }

Until https://github.com/instaclick/php-webdriver implements support for w3c protocol, we need to use the workaround above.

phil-davis commented 5 years ago

@alexdesignworks I have been trying that in https://github.com/owncloud/core/pull/35445 In BEHAT_PARAMS I end up with

"capabilities": {"marionette":null, "browser": "chrome", "platform": "Linux", "name": "owncloud/core - ", "extra_capabilities": {"chromeOptions":{"w3c":false},"maxDuration":"3600"}}

https://drone.owncloud.com/owncloud/core/18090/681

But I get a WebDriver Exception:

    When user "user1" logs in using the webUI                                                           # WebUILoginContext::theUserLogsInUsingTheWebUI()
      WebDriver\Exception\CurlExec: Webdriver http error: 400, payload :{"value":{"error":"invalid argument","message":"invalid argument: missing command parameters","stacktrace":"#0 0x55c24baff299 \u003Cunknown>\n"}} in /drone/src/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:155
      Stack trace:
      #0 /drone/src/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php(132): WebDriver\Exception::factory(-1, 'Webdriver http ...')
      #1 /drone/src/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php(218): WebDriver\AbstractWebDriver->curl('POST', '/click', NULL)
      #2 /drone/src/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/Container.php(224): WebDriver\AbstractWebDriver->__call('click', Array)
      #3 /drone/src/vendor-bin/behat/vendor/behat/mink-selenium2-driver/src/Selenium2Driver.php(781): WebDriver\Container->__call('click', Array)
      #4 /drone/src/vendor-bin/behat/vendor/behat/mink-selenium2-driver/src/Selenium2Driver.php(769): Behat\Mink\Driver\Selenium2Driver->clickOnElement(Object(WebDriver\Element))
      #5 /drone/src/vendor-bin/behat/vendor/behat/mink/src/Element/NodeElement.php(153): Behat\Mink\Driver\Selenium2Driver->click('(//html/.//*[./...')
      #6 /drone/src/tests/acceptance/features/lib/LoginPage.php(71): Behat\Mink\Element\NodeElement->click()
      #7 /tmp/ProxyManagerGeneratedProxy__PM__PageLoginPageGenerated6312256563dff9947346b9792a817253.php(34): Page\LoginPage->loginAs('user1', '1234', 'FilesPage')

Any clues are welcome!

mnocon commented 5 years ago

@phil-davis We're having the same error with our test suite (after setting w3c:false). I'll keep investigating, but for now we will stay with 3.141.59-oxygen to unblock tests.

AlexSkrypnyk commented 5 years ago

@phil-davis I'm getting the same. That is actually comes from Selenium2Driver.php

    private function clickOnElement(Element $element)
    {
        try {
            // Move the mouse to the element as Selenium does not allow clicking on an element which is outside the viewport
            $this->wdSession->moveto(array('element' => $element->getID()));
        } catch (UnknownCommand $e) {
            // If the Webdriver implementation does not support moveto (which is not part of the W3C WebDriver spec), proceed to the click
        }

        $element->click(); // <------ This calls click endpoint
    }

[UPDATE] After more debugging it appears that PHP code works correctly and POST url looks like http://chrome:4444/wd/hub/session/1ea30ec323d3b5f3b1f666005be0c622/element/0.602436616654985-1/click, but this error still persists. I'm wondering if ChromeDriver removed backwards compatibility for non-w3c mode for some methods (maybe even accidentally).

alexsegura commented 5 years ago

Just faced the same error, like @mnocon I fixed it by rollbacking to Chrome 74.

I'm thinking about switching to Chrome Headless, but I'm not sure it would fix the problem.

individual-it commented 5 years ago

There is a bug in the chrome driver https://bugs.chromium.org/p/chromedriver/issues/detail?id=2943 workarounds in facebook/php-webdriver https://github.com/facebook/php-webdriver/pull/640 and nightwatch https://github.com/nightwatchjs/nightwatch/pull/2122

phil-davis commented 5 years ago

Where should we do a "fix" like https://github.com/facebook/php-webdriver/pull/640 In this MinkSelenium2Driver repo, or in https://github.com/instaclick/php-webdriver or?

leymannx commented 5 years ago

Temporarily pinning to Chromedriver 74 seems to be the best workaround for now. See https://github.com/acquia/blt/pull/3708/files/9aa7d8cf28025b957f1e628d9c6de6e63a0a5aac#diff-586532737a140c4f8e2079e2c33fc217

jimarasim commented 5 years ago

When setting w3c to false, use "goog:chromeOptions" instead of "chromeOptions" extra_capabilities: "goog:chromeOptions": w3c: false

Hoping this issue is resolved before Chrome decides to no longer support jsonwire at all.

emodric commented 5 years ago

Is it only me, or this started to work on its own in latest versions of Chrome Driver 75 and even in the new v76?

phil-davis commented 5 years ago

I am using selenium/standalone-chrome-debug in drone-docker. latest of that still fails, and I have to specify 3.141.59-oxygen

emodric commented 5 years ago

Here is a build that works with latest chrome and chrome driver that previously failed: https://travis-ci.org/netgen-layouts/layouts-core/jobs/566143335

Previous failing build: https://travis-ci.org/netgen-layouts/layouts-core/jobs/545643705

shwetaneelsharma commented 5 years ago

@alexdesignworks Thanks a ton for this solution https://github.com/minkphp/MinkSelenium2Driver/issues/293#issuecomment-499458269 However, does it work for headless execution too? I am only able to get it working for browser-based execution

shwetaneelsharma commented 5 years ago

I have got this working on Chrome 75 in headless mode by using the following configuration: `

Behat\MinkExtension:  
    base_url: "your_base_url"  
    browser_name: 'chrome'
    goutte: ~
    javascript_session: selenium2
    selenium2:
      wd_host: http://127.0.0.1:4444/wd/hub
      capabilities:
        browser: chrome
        extra_capabilities:
          chromeOptions:
            args: ['--headless', '--disable-gpu']
            w3c: false`

jimarasim commented 5 years ago

With Chrome 76.0.3809.100 / Chromedriver Version = 76.0.3809.68 '--headless' only works for me if "chromeOptions:" is "goog:chromeOptions"; otherwise, it still runs headed.

marcusanderss commented 4 years ago

The problem is possible 'fix' it by inserting the following 3 lines at the start of the webDriverElement function in Behat, vendor/instaclick/php-webdriver/lib/WebDriver/Container.php: protected function webDriverElement($value) { if (!array_key_exists('ELEMENT', $value)) { $value['ELEMENT'] = $value['element-6066-11e4-a52e-4f735466cecf']; }

robbieaverill commented 4 years ago

Hi everyone, facebook/php-webdriver support for the W3C protocol is nearly ready: https://github.com/facebook/php-webdriver/pull/560#issuecomment-552471707. Would be worth testing an update to use it now? I'm happy to assemble a PR from @tractorcow's fork (to use facebook/php-webdriver) if it would help as well.

AntoscencoVladimir commented 4 years ago

When setting w3c to false, use "goog:chromeOptions" instead of "chromeOptions" extra_capabilities: "goog:chromeOptions": w3c: false

Hoping this issue is resolved before Chrome decides to no longer support jsonwire at all.

Thank you!

Scobee commented 3 years ago

@stof can you please tell me if latest version of Edge & Firefox are supported with Mink selenium2driver, using selenium 4 ?

stof commented 3 months ago

Closing this as we have created the mink/webdriver-classic-driver package.

minkphp / MinkSelenium2Driver

Support the W3C webdriver spec #293

287 should be done first before this work (we should switch to Chrome on Travis, as there is no way to run recent Firefox versions with the JsonWire protocol).