Run tests also with Selenium 3.x

[mvorisek]

There were 3 errored and 2 failing tests, see post below for analysis and how they were fixed.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.73%. Comparing base (5d15043) to head (e84a258). Report is 40 commits behind head on master.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #354 +/- ## ============================================ - Coverage 90.02% 87.73% -2.29% + Complexity 150 149 -1 ============================================ Files 1 1 Lines 421 367 -54 ============================================ - Hits 379 322 -57 - Misses 42 45 +3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mvorisek]

1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame
WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:286
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:426
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:321
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Basic/IFrameTest.php:17

test must be fixed, only integer indexes are accepted, when string id is passed, the following exception is shown:

name key tested, but it is not supported, not handle, nor element itself (might be supported when serialized correctly, but the latest driver does not support it)

WebDriver\Exception\InvalidArgument: invalid argument: 'id' can not be string
(Session info: chrome=84.0.4147.105)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'DX6B48F375QD', ip: '10.8.136.130', os.name: 'Windows 10',
os.arch: 'amd64', os.version: '10.0', java.version: '12.0.2'

---

2) Behat\Mink\Tests\Driver\Js\ChangeEventTest::testSetValueChangeEvent with data set "file" ('the-file', '/home/runner/work/MinkSeleniu...e1.txt', '/home/runner/work/MinkSeleniu...e2.txt')
WebDriver\Exception\InvalidArgument: File not found: /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/../../web-fixtures/file1.txt

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:234
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Container.php:241
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:679
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Element/NodeElement.php:105
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/ChangeEventTest.php:50

the uploaded paths are newly validated if exist and canonical, of course from browser, thus we must use some paths available within the browser container

---

3) Behat\Mink\Tests\Driver\Js\WindowTest::testWindow
WebDriver\Exception\InvalidArgument: missing field `handle` at line 3 column 1

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:269
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:418
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:311
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/WindowTest.php:17

Selenium 3.x + Chrome is working, Firefox is broken - and only handle must be passed, main window handle must be retrived from webdriver first, null/empty is not supported, nor name is supported (/w or /wo handle)

impossible to fix https://stackoverflow.com/questions/25760357/get-title-of-window-without-switching-in-selenium-webdriver, developer must pass window handle instead of window name (which can also be duplicate)

(only) handle support fixed in https://github.com/instaclick/php-webdriver/pull/119

null handle fixed by retriving all handles and using the 1st one (the results is ordered, so the 1st one should always be main windows handle)

---

4) Behat\Mink\Tests\Driver\Custom\TimeoutTest::testInvalidTimeoutSettingThrowsException
Failed asserting that exception of type "\Behat\Mink\Exception\DriverException" is thrown.

/w Selenium 2x the following exception is thrown:

Behat\Mink\Exception\DriverException: Error setting timeout: Unknown wait type: invalid

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:336
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:322
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/tests/Custom/TimeoutTest.php:33

Caused by
WebDriver\Exception\UnknownError: Unknown wait type: invalid

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:319
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:333
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:322
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/tests/Custom/TimeoutTest.php:33

/w Selenium 3.x no exception is thrown with Firefox, in Chrome, exception is thrown, but a different one - WebDriver\Exception\InvalidArgument, test depending on the webdriver impl. removed

---

5) Behat\Mink\Tests\Driver\Js\EventsTest::testRightClick
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'right clicked'
+'single clicked'

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/EventsTest.php:45

natively via webdriver it seems impossible/broken, contextClick method is also not supported, can be probably fixed by using syn.js https://github.com/minkphp/MinkSelenium2Driver/pull/358 for right click - https://github.com/bitovi/syn/blob/v0.15.0/test/mouse_button_test.js#L439 (non upgraded syn.js does not trigger any click $this->trigger($xpath, 'rightClick'); (but 'click' is delivered correctly via syn.js))

$this->wdSession->buttondown(array('button' => 2));
$this->wdSession->buttonup(array('button' => 2));

is not emitting right click either

the only fix seems to migrate to new webdriver actions - https://stackoverflow.com/questions/2171589/how-can-you-do-right-click-using-selenium/19123275#19123275

[aik099]

1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame
WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:286
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:426
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:321
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Basic/IFrameTest.php:17

test must be fixed, only integer indexes are accepted, when string id is passed, the following exception is shown:

name key tested, but it is not supported, not handle, nor element itself (might be supported when serialized correctly, but the latest driver does not support it)

WebDriver\Exception\InvalidArgument: invalid argument: 'id' can not be string
(Session info: chrome=84.0.4147.105)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'DX6B48F375QD', ip: '10.8.136.130', os.name: 'Windows 10',
os.arch: 'amd64', os.version: '10.0', java.version: '12.0.2'

Does Selenium 3 still follow the JsonWire protocol, that allows id parameter to be a string (see https://www.selenium.dev/documentation/legacy/json_wire_protocol/#sessionsessionidframe)?

Otherwise, an upgrade Selenium is only removing stuff instead of adding it :( .

Offtopic: Some time ago, in the https://github.com/minkphp/MinkSelenium2Driver/issues/298 , we're discussing another driver creation (like WebDriverDriver), that will use https://github.com/php-webdriver/php-webdriver (previously developed at Facebook) internally instead of https://github.com/instaclick/php-webdriver, which supports Selenium 2 only (as per the repo's README.md file).

There are also unmerged/closed PRs with attempts to use https://github.com/php-webdriver/php-webdriver in this driver instead.

Maybe we should move in that direction, @stof ?

[aik099]

1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame
WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1

Fixed in https://github.com/minkphp/MinkSelenium2Driver/pull/382 .

2) Behat\Mink\Tests\Driver\Js\ChangeEventTest::testSetValueChangeEvent with data set "file" ('the-file', '/home/runner/work/MinkSeleniu...e1.txt', '/home/runner/work/MinkSeleniu...e2.txt')
WebDriver\Exception\InvalidArgument: File not found: /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/../../web-fixtures/file1.txt

Fixed in https://github.com/minkphp/MinkSelenium2Driver/pull/385 .

3) Behat\Mink\Tests\Driver\Js\WindowTest::testWindow
WebDriver\Exception\InvalidArgument: missing field `handle` at line 3 column 1

Fixed in https://github.com/minkphp/MinkSelenium2Driver/pull/384 .

4) Behat\Mink\Tests\Driver\Custom\TimeoutTest::testInvalidTimeoutSettingThrowsException
Failed asserting that exception of type "\Behat\Mink\Exception\DriverException" is thrown.

Fixed in https://github.com/minkphp/MinkSelenium2Driver/pull/380 , but now test fails again, because no exception is thrown on Selenium 3: https://github.com/minkphp/MinkSelenium2Driver/issues/397 .

5) Behat\Mink\Tests\Driver\Js\EventsTest::testRightClick
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'right clicked'
+'single clicked'

/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/EventsTest.php:45

This will work only through actions and actions not supported in current https://github.com/instaclick/php-webdriver version. So dead end. Issue created: https://github.com/minkphp/MinkSelenium2Driver/issues/381 . Recommended switch to https://github.com/minkphp/webdriver-classic-driver driver if such functionality is needed.

Testing on Selenium 3 was added in https://github.com/minkphp/MinkSelenium2Driver/pull/383.

Closing.

[mvorisek]

Thank you. I am very happy this funcitonality has landed, but I am quite disappointed that I put a lot of efford into this 2 years ago fow nothing. It would be great to warm developers to not invest into this project any energy. I am sorry to say this. Have a great day!

[aik099]

@mvorisek , it wasn't for nothing. Every contribution is welcome. In this case, this was research work, that allowed PRs creation at a later time.

Due to limited maintainer time it might have looked abandoned, but it's not.

[mvorisek]

Thank you for taking care of this project.