search

SeasideSt / Parasol

Testing web apps in Smalltalk using Selenium WebDriver.
MIT License
31 stars 16 forks source link

Parasol self-tests fail randomly #2

Open Rinzwind opened 7 years ago

Rinzwind commented 7 years ago

The Parasol self-tests sometimes suffer from “random” failures. For example, Travis build #58 had one failed test, while the next build had no failing tests (all that changed was the README file).

Rinzwind commented 7 years ago

One source of random failures is tests that do not wait for the browser to complete some action. Some of these should be handled in these commits: https://github.com/SeasideSt/Parasol/commit/1e8cdc253355810ddb539fa5b658b70609ac8a1c, https://github.com/SeasideSt/Parasol/commit/e696c0ea5bcec8002e852ffc0bc21be3b5614f41 and https://github.com/SeasideSt/Parasol/commit/a9c6d66cd559c46e2a069d6a028f7b2e94ce9e64.

Another source of random failures is connection timeouts when communicating with the selenium server. For example in both Travis job 97.2 and job 99.3 there was one test that failed with “ConnectionTimedOut: Data receive timed out.” I did not yet look into how we can fix or avoid those.

Rinzwind commented 7 years ago

Related: issue #7.

Rinzwind commented 7 years ago

The Travis CI jobs for Squeak seem to be particularly prone to the “ConnectionTimedOut: Data receive timed out.” error. In build 172 all jobs passed. In builds 173 through 176, only changes to the README were made, but there are several test failures. All these test failures are due to ConnectionTimedOut, and these all occurred in the two Squeak jobs (except for build 174 which also had one in the GemStone-3.2.15 job).