search

minkphp / MinkSelenium2Driver

Selenium2 (webdriver) driver for Mink framework
MIT License
507 stars 163 forks source link

Also test on Selenium 3 #383

Closed aik099 closed 2 weeks ago

aik099 commented 8 months ago

Observations

Used selenium/standalone-firefox Docker image chooses the Firefox version like so:

Not sure why, but some of the below tests fail only on Firefox 92.

Used build

https://github.com/minkphp/MinkSelenium2Driver/actions/runs/8015977437/job/21897049068?pr=383

Errors

1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1 Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53' System info: host: 'fv-az779-213', ip: '10.1.0.12', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1056-azure', java.version: '1.8.0_292' Driver info: driver.version: unknown

2) Behat\Mink\Tests\Driver\Js\ChangeEventTest::testSetValueChangeEvent with data set "file" ('the-file', '/home/runner/work/MinkSeleniu...e1.txt', '/home/runner/work/MinkSeleniu...e2.txt') WebDriver\Exception\InvalidArgument: File not found: /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/web-fixtures/file1.txt Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53' System info: host: 'fv-az697-197', ip: '10.1.0.44', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1056-azure', java.version: '1.8.0_292' Driver info: driver.version: unknown

3) Behat\Mink\Tests\Driver\Js\WindowTest::testWindow WebDriver\Exception\NoSuchWindow: Unable to locate window: Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53' System info: host: 'fv-az697-197', ip: '10.1.0.44', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1056-azure', java.version: '1.8.0_292' Driver info: driver.version: unknown

Failures:

1) Behat\Mink\Tests\Driver\Custom\TimeoutTest::testInvalidTimeoutSettingThrowsException Failed asserting that exception of type "\Behat\Mink\Exception\DriverException" is thrown.

2) Behat\Mink\Tests\Driver\Js\EventsTest::testRightClick Failed asserting that two strings are equal. --- Expected +++ Actual @@ @@ -'right clicked' +'single clicked'

Doesn't work in both Chrome & Firefox.

Related issues/pull requests:

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.19%. Comparing base (a6cfcfa) to head (edbd07d). Report is 3 commits behind head on master.

:exclamation: Current head edbd07d differs from pull request most recent head a2f3652

Please upload reports for the commit a2f3652 to get more accurate results.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #383 +/- ## ============================================ - Coverage 90.21% 90.19% -0.03% Complexity 168 168 ============================================ Files 1 1 Lines 470 469 -1 ============================================ - Hits 424 423 -1 Misses 46 46 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

aik099 commented 8 months ago

@stof , the PRs for fixing all failing tests on Selenium 3 were created, except the testRightClick. Please review them.

aik099 commented 8 months ago

${{ matrix.selenium_version }} should also be used on line 98 like below, otherwise it will conflict with #385:

          name: logs_php-${{ matrix.php }}_selenium-${{ matrix.selenium_version }}_firefox

Done.

aik099 commented 2 weeks ago

Tests, that weren't failing previously:

1) Behat\Mink\Tests\Driver\Custom\TimeoutTest::testInvalidTimeoutSettingThrowsException
Failed asserting that exception of type "\Behat\Mink\Exception\DriverException" is thrown.
aik099 commented 2 weeks ago

Merging, because created dedicated issues for fixing failing tests.