Starting with enabled test_workers seems to not use selenium-webdriver correctly. When running a test session with them enabled, the parallel tests do not seem to correctly want to spawn chromedrivers (only the first worker gets one spawned):
Starting ChromeDriver on port 9515...
test.js ⚠ Failed to connect to ChromeDriver on localhost with port 9515.
Whereas adding --parallel to the CLI call seems to fix it. Notably, in verbose the tests report (using --parallel):
Starting ChromeDriver on auto-generated port...
The strange thing is that there seems to be some race condition when only test_workers are enabled. Making this extremely flaky. In some situations it passes checks (likely connects to the default port), one or more tests start running but it then has a ECONNRESET (socket hang up) when the first driver process stops: but only sometimes. Sometimes the driver is kept alive between starting and stopping test suites: super weird. Setting parallel_process_delay makes it less flaky and makes it always fail.
I did not add an example script because this seems independent on test-suite contents.
Describe the bug
Starting with enabled
test_workers
seems to not use selenium-webdriver correctly. When running a test session with them enabled, the parallel tests do not seem to correctly want to spawn chromedrivers (only the first worker gets one spawned):Whereas adding
--parallel
to the CLI call seems to fix it. Notably, in verbose the tests report (using--parallel
):The strange thing is that there seems to be some race condition when only test_workers are enabled. Making this extremely flaky. In some situations it passes checks (likely connects to the default port), one or more tests start running but it then has a
ECONNRESET
(socket hang up) when the first driver process stops: but only sometimes. Sometimes the driver is kept alive between starting and stopping test suites: super weird. Settingparallel_process_delay
makes it less flaky and makes it always fail.I did not add an example script because this seems independent on test-suite contents.
Configuration
nightwatch.conf.js (with `test_workers` enabled)
```js module.exports = { 'src_folders' : ['nightwatch/tests'], 'page_objects_path': 'nightwatch/pages', 'output_timestamp': true, 'live_output': true, 'webdriver' : { 'start_process': true, }, 'test_settings' : { 'default' : { 'launch_url': 'http://localhost:8888/cirrus', 'desiredCapabilities': { 'browserName': 'chrome', 'goog:chromeOptions': { 'w3c': true, }, 'globals': { 'waitForConditionTimeout': 10000, }, }, 'screenshots': { 'enabled': false, 'path': './screenshots', 'on_error': true, 'on_failure': true, }, }, }, // Parallel settings 'parallel_process_delay': 200, 'test_workers': { 'enabled': true, 'workers': 4, }, } ```
nightwatch.conf.js (with `test_workers` disabled)
```js module.exports = { 'src_folders' : ['nightwatch/tests'], 'page_objects_path': 'nightwatch/pages', 'output_timestamp': true, 'live_output': true, 'webdriver' : { 'start_process': true, }, 'test_settings' : { 'default' : { 'launch_url': 'http://localhost:8888/cirrus', 'desiredCapabilities': { 'browserName': 'chrome', 'goog:chromeOptions': { 'w3c': true, }, 'globals': { 'waitForConditionTimeout': 10000, }, }, 'screenshots': { 'enabled': false, 'path': './screenshots', 'on_error': true, 'on_failure': true, }, }, }, // Parallel settings 'parallel_process_delay': 200, 'test_workers': { 'enabled': false, 'workers': 4, }, } ```
Your Environment
nightwatch --version
npm --version
yarn --version
node --version