Tests not running in parallel after release v1.7.6

[chriscuba23]

Describe the bug

Prior to release v1.7.6 tests belonging under the same folder could run concurrently. Now they do not run in parallel even if I use the --parallel tag . The tests run sequentially

Sample test

Test path is

/tests/ci_tests/mt1.js /tests/ci_tests/mt2.js
/tests/ci_tests/mt3.js /tests/ci_tests/mt4.js

Run with command node nightwatch -e chrome .\tests\ci_tests --parallel --verbose

Verbose output

Tests finishing one by one even if I use --parallel tag or workers

debug.log

```txt Running Navigate to MT 1: → Running [beforeEach]: → Completed [beforeEach]. → Running command: url ('https://stage.marinetraffic.com/') Request POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/url { url: 'https://stage.marinetraffic.com/' } Response 200 POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/url (2712ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: null } → Completed command: url ('https://stage.marinetraffic.com/') (2713ms) → Running command: windowMaximize () Request POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/window/current/maximize {} Response 200 POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/window/current/maximize (225ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: { height: 1056, width: 1936, x: -8, y: 32 } } → Completed command: windowMaximize () (227ms) → Running command: waitForElementVisible ('body') Request POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/elements { using: 'css selector', value: 'body' } Response 200 POST /session/8f950e8ef870a3f1eac8a8bf600e9d87/elements (86ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: [ { ELEMENT: '0.026731140659515207-1' } ] } Request GET /session/8f950e8ef870a3f1eac8a8bf600e9d87/element/0.026731140659515207-1/displayed Response 200 GET /session/8f950e8ef870a3f1eac8a8bf600e9d87/element/0.026731140659515207-1/displayed (174ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: true } √ Element was visible after 263 milliseconds. → Completed command: waitForElementVisible ('body') (264ms) → Running command: verify.title ('MarineTraffic: Global Ship Tracking Intelligence | AIS Marine Traffic') → Running command: title ([Function]) Request GET /session/8f950e8ef870a3f1eac8a8bf600e9d87/title Response 200 GET /session/8f950e8ef870a3f1eac8a8bf600e9d87/title (257ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: 'MarineTraffic: Global Ship Tracking Intelligence | AIS Marine Traffic' } √ Testing if the page title equals 'MarineTraffic: Global Ship Tracking Intelligence | AIS Marine Traffic' (259ms) → Completed command: verify.title ('MarineTraffic: Global Ship Tracking Intelligence | AIS Marine Traffic') (260ms) → Completed command: title ([Function]) (259ms) → Running [afterEach]: → Completed [afterEach]. OK. 2 assertions passed. (3.468s) Running end test: → Running [beforeEach]: → Completed [beforeEach]. → Running command: closeWindow () → Running command: window ('DELETE', [Function]) Request DELETE /session/8f950e8ef870a3f1eac8a8bf600e9d87/window Response 200 DELETE /session/8f950e8ef870a3f1eac8a8bf600e9d87/window (4157ms) { sessionId: '8f950e8ef870a3f1eac8a8bf600e9d87', status: 0, value: [] } → Completed command: closeWindow () (4173ms) → Completed command: window ('DELETE', [Function]) (4160ms) → Running command: end () → Running command: session ('delete', [Function]) Request DELETE /session/8f950e8ef870a3f1eac8a8bf600e9d87 Response 200 DELETE /session/8f950e8ef870a3f1eac8a8bf600e9d87 (56ms) { value: null } → Completed command: end () (67ms) → Completed command: session ('delete', [Function]) (57ms) → Running [afterEach]: → Completed [afterEach]. No assertions ran. → Running [after]: → Completed [after]. ```

Configuration

debug info

```js // Autogenerated by Nightwatch // Refer to the online docs for more details: https://nightwatchjs.org/gettingstarted/configuration/ const Services = {}; loadServices(); module.exports = { // An array of folders (excluding subfolders) where your tests are located; // if this is not specified, the test source must be passed as the second argument to the test runner. src_folders: [], // See https://nightwatchjs.org/guide/working-with-page-objects/ page_objects_path: '', // See https://nightwatchjs.org/guide/extending-nightwatch/#writing-custom-commands custom_commands_path: '', // See https://nightwatchjs.org/guide/extending-nightwatch/#writing-custom-assertions custom_assertions_path: '', // See https://nightwatchjs.org/guide/#external-globals globals_path : '', webdriver: {}, test_settings: { default: { test_workers : {"enabled" : true, "workers" : 10}, disable_error_log: false, launch_url: 'https://nightwatchjs.org', screenshots: { enabled: false, path: 'screens', on_failure: true }, desiredCapabilities: { browserName : 'firefox' }, webdriver: { start_process: true, server_path: (Services.geckodriver ? Services.geckodriver.path : '') } }, firefox: { desiredCapabilities : { browserName : 'firefox', alwaysMatch: { acceptInsecureCerts: true, 'moz:firefoxOptions': { args: [ // '-headless', // '-verbose' ] } } }, webdriver: { start_process: true, port: 4444, server_path: (Services.geckodriver ? Services.geckodriver.path : ''), cli_args: [ // very verbose geckodriver logs // '-vv' ] } }, chrome: { desiredCapabilities : { browserName : 'chrome', 'goog:chromeOptions' : { // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/ // // This tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78) w3c: false, args: [ //'--no-sandbox', //'--ignore-certificate-errors', //'--allow-insecure-localhost', //'--headless' ] } }, webdriver: { start_process: true, port: 9515, server_path: (Services.chromedriver ? Services.chromedriver.path : ''), cli_args: [ // --verbose ] } }, edge: { desiredCapabilities : { browserName : 'MicrosoftEdge', 'ms:edgeOptions' : { w3c: false, // More info on EdgeDriver: https://docs.microsoft.com/en-us/microsoft-edge/webdriver-chromium/capabilities-edge-options args: [ //'--headless' ] } }, webdriver: { start_process: true, // Download msedgedriver from https://docs.microsoft.com/en-us/microsoft-edge/webdriver-chromium/ // and set the location below: server_path: '', cli_args: [ // --verbose ] } }, ////////////////////////////////////////////////////////////////////////////////// // Configuration for when using the browserstack.com cloud service | // | // Please set the username and access key by setting the environment variables: | // - BROWSERSTACK_USER | // - BROWSERSTACK_KEY | // .env files are supported | ////////////////////////////////////////////////////////////////////////////////// browserstack: { selenium: { host: 'hub-cloud.browserstack.com', port: 443 }, // More info on configuring capabilities can be found on: // https://www.browserstack.com/automate/capabilities?tag=selenium-4 desiredCapabilities: { 'bstack:options' : { userName: '${BROWSERSTACK_USER}', accessKey: '${BROWSERSTACK_KEY}', } }, disable_error_log: true, webdriver: { timeout_options: { timeout: 15000, retry_attempts: 3 }, keep_alive: true, start_process: false } }, 'browserstack.local': { extends: 'browserstack', desiredCapabilities: { 'browserstack.local': true } }, 'browserstack.chrome': { extends: 'browserstack', desiredCapabilities: { browserName: 'chrome', chromeOptions : { w3c: false } } }, 'browserstack.firefox': { extends: 'browserstack', desiredCapabilities: { browserName: 'firefox' } }, 'browserstack.ie': { extends: 'browserstack', desiredCapabilities: { browserName: 'internet explorer', browserVersion: '11.0' } }, 'browserstack.safari': { extends: 'browserstack', desiredCapabilities: { browserName: 'safari' } }, 'browserstack.local_chrome': { extends: 'browserstack.local', desiredCapabilities: { browserName: 'chrome' } }, 'browserstack.local_firefox': { extends: 'browserstack.local', desiredCapabilities: { browserName: 'firefox' } }, ////////////////////////////////////////////////////////////////////////////////// // Configuration for when using the Selenium service, either locally or remote, | // like Selenium Grid | ////////////////////////////////////////////////////////////////////////////////// selenium_server: { // Selenium Server is running locally and is managed by Nightwatch selenium: { start_process: true, port: 4444, server_path: (Services.seleniumServer ? Services.seleniumServer.path : ''), cli_args: { 'webdriver.gecko.driver': (Services.geckodriver ? Services.geckodriver.path : ''), 'webdriver.chrome.driver': (Services.chromedriver ? Services.chromedriver.path : '') } } }, 'selenium.chrome': { extends: 'selenium_server', desiredCapabilities: { browserName: 'chrome', chromeOptions : { w3c: false } } }, 'selenium.firefox': { extends: 'selenium_server', desiredCapabilities: { browserName: 'firefox', 'moz:firefoxOptions': { args: [ // '-headless', // '-verbose' ] } } } } }; function loadServices() { try { Services.seleniumServer = require('selenium-server'); } catch (err) {} try { Services.chromedriver = require('chromedriver'); } catch (err) {} try { Services.geckodriver = require('geckodriver'); } catch (err) {} } ```

Your Environment

Executable	Version
nightwatch --version	version: 1.7.6
npm --version	6.9.0
yarn --version	VERSION
node --version	v10.16.3

Browser driver	Version
Google Chrome	Version 91.0.4472.106 (Official Build) (64-bit)

OS	Version
NAME	VERSION
Windows	10

[beatfactor]

Do you run selenium separately or managed by nightwatch?

[beatfactor]

Also, which version are you updating from?

[chriscuba23]

Hello and thank you for the response. I have tried both start_process = true and start_process = false and the result is the same. The upgrade is from 1.6.4 to 1.7.3

[beatfactor]

Perhaps you can try using the generated config? If you delete/rename your existing config file, Nightwatch will create one by default for you. You seem to be using a very old config format and there must be a bug somewhere in parsing it.

[beatfactor]

@chriscuba23 I'm not really able to reproduce your scenario, but if I remove the test_workers settings and then try to run by passing the --parallel flag, there is indeed the issue of not getting the automatic worker count correctly. Is that the issue you are seeing? If test_workers are specified, then it works fine.

[chriscuba23]

Please let me check the conf file and I will get back to you as soon as possible

[chriscuba23]

@beatfactor closing this one since the new configuration file did the trick. Thank you

[chriscuba23]

@beatfactor reopening this bug for release 1.7.6 this time I've updated the conf with the new one

On the other hand parallelism works fine when you use tags like '-a test1.js -a test2.js' instead of a path

[beatfactor]

can you post the verbose output in the place of debug.log?

[chriscuba23]

Of course. @beatfactor you will find debug.log updated with the verbose output

[beatfactor]

@chriscuba23 worse fine for me. Please check your config and any other external tools which might affect the nightwatch settings (such as environment variables) or try to reproduce the issue by creating a simpler scenario with a minimal config.

[chriscuba23]

can you please post the command that you run in order to trigger the tests in parallel?

[beatfactor]

There are several ways in which you can do that. In order for me to help you, I need to be able to reproduce the issue successfully, which I can't. So please try to create a scenario with minimal configuration which reproduces the issue.

[chriscuba23]

ok will create a new conf and let you know. Thank you!

[chriscuba23]

@beatfactor I've updated the conf and verbose output log. The configuration file is brand new and auto created

[beatfactor]

I don't see test_workers being specified anywhere in your config.

[chriscuba23]

@beatfactor updated with test_workers. Still doesnt work

[beatfactor]

@vaibhavsingh97 @Dharin-shah @gravityvi does any of you have access to a windows machine and could verify this? I am out of ideas.

[vaibhavsingh97]

Checking, will update the issue with the result

On Sat, 26 Jun, 2021, 5:21 pm Andrei Rusu, @.***> wrote:

@vaibhavsingh97 https://github.com/vaibhavsingh97 @Dharin-shah https://github.com/Dharin-shah @gravityvi https://github.com/gravityvi does any of you have access to a windows machine and could verify this? I am out of ideas.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nightwatchjs/nightwatch/issues/2748#issuecomment-868990156, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCNK2QBFIXVBEOFIUSVNDLTUW5NLANCNFSM463V5Q5Q .

[vaibhavsingh97]

Tried on windows 10 machine, it's working perfectly fine for me

[chriscuba23]

Can you post your node version please? Updated to v14.17.1 and still no luck

[vaibhavsingh97]

Yes, the node version is v14.17.1

[chriscuba23]

@vaibhavsingh97 tried with test_workers outside of

test_settings: { default:{ } }

but still no luck

[vaibhavsingh97]

https://user-images.githubusercontent.com/8705386/123620436-dabbd880-d827-11eb-8b51-10820b8e6c23.mp4

[chriscuba23]

@vaibhavsingh97 this indeed works in parallel, but you execute two different js files, not a whole folder

[vaibhavsingh97]

@chriscuba23 Oops!! Yes, can confirm that it is broken. Instead of running parallel, it is running single test files.

[chriscuba23]

@vaibhavsingh97 thanx for verifying. I am leaving it open then

[beatfactor]

@chriscuba23 Should be fixed in v1.7.7.

[chriscuba23]

@beatfactor @vaibhavsingh97 Thank you for your help!

[chriscuba23]

@beatfactor this does not work with release 1.7.8 either