search

giorgiosironi / phpunit-selenium

Selenium RC integration for PHPUnit
http://www.phpunit.de/
Other
600 stars 271 forks source link

After upgrading from PHPUnit 8 to 9, the number of tests exploded #457

Open validaide-mark-bijl opened 2 years ago

validaide-mark-bijl commented 2 years ago

Hi!

Well, first of all, thanks for a great library! I got introduced to it after we adopted a no-longer maintained OS library (https://github.com/egeloen/ivory-google-map). As that library is quite outdated, we've been working step by step to upgrade its dependencies. This time we wanted to upgrade PHPUnit and PHPUnit Selenium from 8 to 9. However, after we did so the number of tests went from 1810 to 15722.

BEFORE:

PHPUnit 8.5.31 by Sebastian Bergmann and contributors.

.............................................................   61 / 1810 (  3%)

AFTER:

PHPUnit 9.5.26 by Sebastian Bergmann and contributors.

Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

...........................................................    59 / 15722 (  0%)

When I ran it verbose, I noticed things like these:

33) Ivory\Tests\GoogleMap\Helper\Functional\CompoundFunctionalTest::__construct
The Selenium Server is not active on host selenium at port 4444.

/Users/markbijl/projects/google-map/vendor/phpunit/phpunit-selenium/PHPUnit/Extensions/Selenium2TestCase.php:345
/Users/markbijl/projects/google-map/vendor/phpunit/phpunit-selenium/PHPUnit/Extensions/Selenium2TestCase.php:384
/Users/markbijl/projects/google-map/vendor/phpunit/phpunit-selenium/PHPUnit/Extensions/Selenium2TestCase.php:360
phpvfscomposer:///Users/markbijl/projects/google-map/vendor/phpunit/phpunit/phpunit:97

Notice the test CompoundFunctionalTest::__construct where the class 'CompoundFunctionalTest' extends the Selenium2TestCase. But... it seems like the constructor is identified as a test, same for every other method of the test classes that extend the Selenium2TestCase.

Basically, it seems like every method of every test class is being identified as a test case. Now I admit that my knowledge of this whole library is limited (getting to know it better every day) and I am still trawling through the PHPUnit setup thinking that perhaps there is some kind of magic happening that is causing every class method to be identified as a test, but... I also though to check here to see if perhaps this is something related to the upgrade of PHPUnit Selenium from 8 to 9.

Any tips, hints, feedback... all much appreciated!

Thanks, Mark

p.s. this is our fork: https://github.com/validaide/google-map

paprykasz commented 1 year ago

+1 After quick research it seems that fix is included in master https://github.com/giorgiosironi/phpunit-selenium/commit/1e9ba6087ab055643de0c6d58eb659fd78807dd2 but not released. I was looking for some quick port but library seems abandoned.