Closed phil-davis closed 5 years ago
Merging #158 into master will not change coverage. The diff coverage is
n/a
.
@@ Coverage Diff @@
## master #158 +/- ##
=========================================
Coverage 78.71% 78.71%
Complexity 214 214
=========================================
Files 26 26
Lines 916 916
=========================================
Hits 721 721
Misses 195 195
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 2555ee7...90e5b77. Read the comment docs.
On my local development system this scenario would fail about 25% of the time:
make test-acceptance-webui BEHAT_FEATURE=tests/acceptance/features/webUIManageUsersGroups/addUsers.feature:74
Watching the browser, I could see that I did not successfully click off "Set password for new users". And so the password box stayed and the email box was not displayed for creating a new user.
And so the test fails with:
Examples:
| username | comment |
| guiusr1 | simple user-name |
WebDriver\Exception\ElementNotVisible: element not visible
(Session info: chrome=67.0.3396.87)
(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.15.0-46-generic x86_64) in /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:155
Stack trace:
#0 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php(157): WebDriver\Exception::factory(11, 'element not vis...')
#1 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php(218): WebDriver\AbstractWebDriver->curl('POST', '/value', Array)
#2 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/instaclick/php-webdriver/lib/WebDriver/Container.php(224): WebDriver\AbstractWebDriver->__call('postValue', Array)
#3 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/behat/mink-selenium2-driver/src/Selenium2Driver.php(678): WebDriver\Container->__call('postValue', Array)
#4 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/behat/mink/src/Element/NodeElement.php(105): Behat\Mink\Driver\Selenium2Driver->setValue('(//html/.//*\n[s...', 'guiusr1@ownclou...')
#5 /home/phil/git/owncloud/core/apps-external/user_management/vendor-bin/behat/vendor/behat/mink/src/Element/TraversableElement.php(160): Behat\Mink\Element\NodeElement->setValue('guiusr1@ownclou...')
#6 /home/phil/git/owncloud/core/apps-external/user_management/tests/acceptance/features/lib/UsersPage.php(404): Behat\Mink\Element\TraversableElement->fillField('newemail', 'guiusr1@ownclou...')
After adding this sleep(1)
I ran the test 10 times and I does not fail.
Part of issue https://github.com/owncloud/core/issues/34689
There are webUI tests on the users page that adjust the settings and then check that columns are visible or not, or try to enter password or email address etc. These tests have become intermittently unreliable. I think faster test systems will see this timing issue more often.
After investigation, I realized that the settings menu on the users page opens with an animation. The observed behavior is that if the test tries to click a setting checkbox before the animation is finished, then the click is not effective, but does not return and error.
However, as soon as the test clicks to open the settings menu,
isVisible()
on the settings menu returns true - although the menu might still be animating itself.Basically it's a PITA. We could design some sort of JS query to send in a loop to try and see if the menu is still in an "animating" state. Or we can repeatedly query the size/position of the menu div and wait until it has been "steady" for "a while". They are the 2 common approaches that others take to try and cope with waiting for animations. That will all take time to work out the gory details.
At the moment this is causing significant percentages of "random" test fails, which is not "a good thing". So put in a 1 second sleep (which I hate to do)