SeleniumHQ / docker-selenium

Provides a simple way to run Selenium Grid with Chrome, Firefox, and Edge using Docker, making it easier to perform browser automation
http://www.selenium.dev/docker-selenium/
Other
7.93k stars 2.51k forks source link

[🐛 Bug]: Unable to Run Selenium test in Parallel/Concurrently in Docker Container #1533

Closed ramgiteng closed 2 years ago

ramgiteng commented 2 years ago

What happened?

Objective:

To execute Selenium tests across a number of chrome browsers in parallel within a Docker container consisting of chrome, chromedriver, and the Selenium server

  1. I was able to successfully run the docker container in the background. (Second command in the next section)
  2. After pointing my Selenium script (written in .js) to http://localhost:4444/wd/hub, I was able to successfully execute my test script for only one iteration. (Thirdcommand in the next section)

The selenium script navigates to a page, gets the text of an element, and writes the text to console.

Expected Output:

For 3 iterations/sessions, the following is the expected output. Text is a placeholder for the actual text in element. Text Text Text

Actual Output:

Text

Explanation:

As shown above, I was able to only run one browser session within the container despite setting SE_NODE_MAX_SESSIONS=3 and SE_NODE_OVERRIDE_MAX_SESSIONS=true. I would like to run across 3 browsers.

We are able to successfully execute for one iteration.

Details:

OS Version: CentOS 7.5 No. of Cores: 4 Memory: 16 GB Storage: 120 GB Docker version: 1.13.1 Docker-Selenium image tag/version: 4.1.3-20220327

But, how can we run a Selenium test across a number of parallel browsers concurrently within the docker container? Please let me know if you need further details. Thanks for the support!

Command used to start Selenium Grid with Docker

Docker image:
docker pull selenium/standalone-chrome:4.1.3-20220327

To start Selenium Grid with Docker: 
docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" -e SE_NODE_MAX_SESSIONS=4 -e SE_NODE_OVERRIDE_MAX_SESSIONS=true -e START_XVFB=false --name="seleniumchrome" selenium/standalone-chrome

To start my Selenium script on my local machine (This script navigates to a page and gets the text of an element :
node ./SeleniumScript.js

Relevant log output

Docker Container logs:

Setting up SE_NODE_GRID_URL...
Selenium Grid Standalone configuration
...
...
max-sessions=3
...
...
Starting Selenium Grid Standalone...
...
INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
....
....
INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion" : "99.0", "platformName": "Linux"} 3 times
....
INFO [Standalone.execute] - Started Selenium Standalone 4.1.3 (revision 7b1ebf28ef): http:<IP>:4444

Operating System

CentOS 7.5

Docker Selenium version (tag)

4.1.3-20220327

ramgiteng commented 2 years ago

Note: Please refer to the below logs from the Docker container

2022-03-29 18:05:00,708 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2022-03-29 18:05:00,710 INFO supervisord started with pid 8
2022-03-29 18:05:01,713 INFO spawned: 'xvfb' with pid 10
2022-03-29 18:05:01,715 INFO spawned: 'vnc' with pid 11
2022-03-29 18:05:01,716 INFO spawned: 'novnc' with pid 12
2022-03-29 18:05:01,718 INFO spawned: 'selenium-standalone' with pid 13
2022-03-29 18:05:01,722 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-29 18:05:01,722 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-29 18:05:01,722 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-29 18:05:01,722 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-29 18:05:01,722 INFO exited: xvfb (exit status 0; expected)
2022-03-29 18:05:01,722 INFO exited: vnc (exit status 0; expected)
2022-03-29 18:05:01,724 INFO exited: novnc (exit status 0; expected)
Setting up SE_NODE_GRID_URL...
Selenium Grid Standalone configuration: 
[network]
relax-checks = true
[node]
session-timeout = "300"
override-max-sessions = true
detect-drivers = false
max-sessions = 3
[[node.driver-configuration]]
display-name = "chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "99.0", "platformName": "Linux"}'
max-sessions = 3
Starting Selenium Grid Standalone...
18:05:02.076 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
18:05:02.079 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
18:05:02.584 INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
18:05:02.585 WARN [NodeOptions.getSessionFactories] - Overriding max recommended number of 4 concurrent sessions. Session stability and reliability might suffer!
18:05:02.585 WARN [NodeOptions.getSessionFactories] - One browser session is recommended per available processor. Safari is always limited to 1 session per host.
18:05:02.585 WARN [NodeOptions.getSessionFactories] - Overriding this value for Internet Explorer is not recommended. Issues related to parallel testing with Internet Explored won't be accepted.
18:05:02.585 WARN [NodeOptions.getSessionFactories] - Double check if enabling 'override-max-sessions' is really needed
18:05:02.631 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "99.0","platformName": "Linux"} 3 times
18:05:02.644 INFO [Node.<init>] - Binding additional locator mechanisms: id, name, relative
18:05:02.660 INFO [GridModel.setAvailability] - Switching node 803fa2e0-6e82-4105-801d-1b175ea3f7cd (uri: http://"IPaddr":4444) from DOWN to UP
18:05:02.661 INFO [LocalDistributor.add] - Added node 803fa2e0-6e82-4105-801d-1b175ea3f7cd at http://"IPaddr":4444. Health check every 120s
18:05:02.778 INFO [Standalone.execute] - Started Selenium Standalone 4.1.3 (revision 7b1ebf28ef): http://"IPaddr":4444
unickq commented 2 years ago

still there, as https://github.com/SeleniumHQ/docker-selenium/issues/1462 :(

ramgiteng commented 2 years ago

@unickq based on your old ticket, I did not experience the same issue in terms of max-sessions param being set to 1.

In my case as you can see in the logs, the max-sessions=3 which is correct based on the env vars I set in the docker command. However, when I run node SeleniumScript.json my local machine (pointing the script to localhost:4444/wd/hub), I only get the output for one session.

I ran: docker logs {container}

It showed that only one session/browser was executed in the container.

unickq commented 2 years ago

@ramgiteng yes, agree. I uploaded the wrong screenshot. Here are correct - 8 threads, but tests go one by one

image image
ramgiteng commented 2 years ago

@unickq ok, this is strange. So, your logs suggest that the execution is happening rather sequentially . I'm not getting even that (i.e. not even one by one). Only one session runs and completes in my container.

This is how I'm doing it.

To start Selenium Grid with Docker: docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" -e SE_NODE_MAX_SESSIONS=4 -e SE_NODE_OVERRIDE_MAX_SESSIONS=true -e START_XVFB=false --name="seleniumchrome" selenium/standalone-chrome

To start my Selenium script on my local machine: node ./SeleniumScript.js

Do I need to add any configurations in the Selenium script or add anything additional in the docker command itself? May I ask how you are running your Selenium script after running the docker-selenium container?

unickq commented 2 years ago

@ramgiteng I use wdio with default options. Default params, host port. selenium/standalone-chrome:4.1.3 runs on CirleCI as a Docker executor.

Looks like the thing I see is not related to your problem, sorry.

ramgiteng commented 2 years ago

FYI - Below are the logs of the docker container running Selenium server after executing node SeleniumScript.js on my local machine

2022-03-30 20:05:25,616 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2022-03-30 20:05:25,617 INFO supervisord started with pid 9
2022-03-30 20:05:26,620 INFO spawned: 'xvfb' with pid 11
2022-03-30 20:05:26,622 INFO spawned: 'vnc' with pid 12
2022-03-30 20:05:26,624 INFO spawned: 'novnc' with pid 13
2022-03-30 20:05:26,627 INFO spawned: 'selenium-standalone' with pid 14
2022-03-30 20:05:26,629 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-30 20:05:26,629 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-30 20:05:26,629 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-30 20:05:26,630 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-30 20:05:26,630 INFO exited: xvfb (exit status 0; expected)
2022-03-30 20:05:26,630 INFO exited: vnc (exit status 0; expected)
2022-03-30 20:05:26,630 INFO exited: novnc (exit status 0; expected)
Setting up SE_NODE_GRID_URL...
Selenium Grid Standalone configuration: 
[network]
relax-checks = true
[node]
session-timeout = "300"
override-max-sessions = true
detect-drivers = false
max-sessions = 3
[[node.driver-configuration]]
display-name = "chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "99.0", "platformName": "Linux"}'
max-sessions = 3
Starting Selenium Grid Standalone...
20:05:26.999 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
20:05:27.004 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
20:05:27.526 INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
20:05:27.527 WARN [NodeOptions.getSessionFactories] - Overriding max recommended number of 4 concurrent sessions. Session stability and reliability might suffer!
20:05:27.528 WARN [NodeOptions.getSessionFactories] - One browser session is recommended per available processor. Safari is always limited to 1 session per host.
20:05:27.528 WARN [NodeOptions.getSessionFactories] - Overriding this value for Internet Explorer is not recommended. Issues related to parallel testing with Internet Explored won't be accepted.
20:05:27.528 WARN [NodeOptions.getSessionFactories] - Double check if enabling 'override-max-sessions' is really needed
20:05:27.562 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "99.0","platformName": "Linux"} 3 times
20:05:27.574 INFO [Node.<init>] - Binding additional locator mechanisms: relative, id, name
20:05:27.595 INFO [GridModel.setAvailability] - Switching node e0120dba-702d-433c-8682-88fbe43bea07 (uri: http://IP:4444) from DOWN to UP
20:05:27.596 INFO [LocalDistributor.add] - Added node e0120dba-702d-433c-8682-88fbe43bea07 at http://IP:4444. Health check every 120s
20:05:27.705 INFO [Standalone.execute] - Started Selenium Standalone 4.1.3 (revision 7b1ebf28ef): http://IP:4444
20:05:31.131 INFO [LocalDistributor.newSession] - Session request received by the distributor: 
 [Capabilities {browserName: chrome, chromeOptions: {args: [--headless, --disable-dev-shm-usage, --disable-extensions, --no-sandbox]}, useAutomationExtension: false}, Capabilities {browserName: chrome}, Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --disable-dev-shm-usage, --disable-extensions, --no-sandbox]}}]
Starting ChromeDriver 99.0.4844.51 (d537ec02474b5afe23684e7963d538896c63ac77-refs/branch-heads/4844@{#875}) on port 37780
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
20:05:31.558 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
20:05:31.575 INFO [LocalDistributor.newSession] - Session created by the distributor. Id: 80050d2dff097d620bbb8d07a7f191e1, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 99.0.4844.84, chrome: {chromedriverVersion: 99.0.4844.51 (d537ec02474b5..., userDataDir: /tmp/.com.google.Chrome.BCQSvl}, chromeOptions: {args: [--headless, --disable-dev-shm-usage, --disable-extensions, --no-sandbox]}, goog:chromeOptions: {debuggerAddress: localhost:45265}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:cdp: ws://IP:4444/sessio..., se:cdpVersion: 99.0.4844.84, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, useAutomationExtension: false, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
20:05:33.272 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local session map, Id: 80050d2dff097d620bbb8d07a7f191e1
diemol commented 2 years ago

Can you please share the code from your SeleniumScript.js script?

unickq commented 2 years ago

Tried on my mac, session not started: docker run -d -p 4444:4444 -p 7900:7900 selenium/standalone-chrome:4.1.3

2022-03-31 10:51:41,322 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2022-03-31 10:51:41,329 INFO supervisord started with pid 13
2022-03-31 10:51:42,364 INFO spawned: 'xvfb' with pid 19
2022-03-31 10:51:42,375 INFO spawned: 'vnc' with pid 21
2022-03-31 10:51:42,382 INFO spawned: 'novnc' with pid 23
2022-03-31 10:51:42,390 INFO spawned: 'selenium-standalone' with pid 25
Setting up SE_NODE_GRID_URL...
2022-03-31 10:51:42,572 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-31 10:51:42,572 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-31 10:51:42,572 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2022-03-31 10:51:42,572 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
Selenium Grid Standalone configuration: 

[network]
relax-checks = true

[node]
session-timeout = "300"
override-max-sessions = false
detect-drivers = false
max-sessions = 1

[[node.driver-configuration]]
display-name = "chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "99.0", "platformName": "Linux"}'
max-sessions = 1

Starting Selenium Grid Standalone...
10:51:45.506 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
10:51:45.533 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
10:51:49.268 INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
10:51:49.548 INFO [NodeOptions.report] - Adding chrome for {"browserVersion": "99.0","se:noVncPort": 7900,"browserName": "chrome","platformName": "Linux","se:vncEnabled": true} 1 times
10:51:49.622 INFO [Node.<init>] - Binding additional locator mechanisms: name, relative, id
10:51:49.742 INFO [GridModel.setAvailability] - Switching node 100341c0-0b12-4cc9-a033-0b2b5ee83b1b (uri: http://172.17.0.2:4444) from DOWN to UP
10:51:49.747 INFO [LocalDistributor.add] - Added node 100341c0-0b12-4cc9-a033-0b2b5ee83b1b at http://172.17.0.2:4444. Health check every 120s
10:51:50.443 INFO [Standalone.execute] - Started Selenium Standalone 4.1.3 (revision 7b1ebf28ef): http://172.17.0.2:4444
10:51:56.349 INFO [LocalDistributor.newSession] - Session request received by the distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--window-size=1920,1080, --disable-extensions, --disable-plugins, --disable-dev-shm-usage, --disable-blink-features, --enable-automation], excludeSwitches: [enable-automation]}}]
Starting ChromeDriver 99.0.4844.51 (d537ec02474b5afe23684e7963d538896c63ac77-refs/branch-heads/4844@{#875}) on port 56898
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
[1648723916.800][SEVERE]: bind() failed: Cannot assign requested address (99)
ChromeDriver was started successfully.

and then it just stuck

diemol commented 2 years ago

@unickq what type of Mac do you have? Intel or M1?

ramgiteng commented 2 years ago

@diemol Thanks for your reply. As requested, please take a look at the code for SeleniumScript.js

const webdriver = require('selenium-webdriver');
const { By, until } = require('selenium-webdriver');

const chromeCapabilities = webdriver.Capabilities.chrome();
chromeCapabilities.set('chromeOptions', { 'args':[ '--headless','--disable-dev-shm-usage','--disable-extensions','--no-sandbox']});
chromeCapabilities.set('useAutomationExtension', false);

const driver = new webdriver.Builder().withCapabilities(chromeCapabilities).usingServer('http://localhost:4444/wd/hub').build();

console.log('Starting chrome');
(async () => {
await driver.get('http://www.lambdatest.com');
const text = await driver.wait(until.elementIsVisible(driver.findElement(By.xpath("//h1[contains(text(),'Cross Browser')]")),20000)).getText();

console.log("Text is: " + text);
driver.quit();
})(); 
diemol commented 2 years ago

@ramgiteng I do not see how that code is running 3 sessions in parallel? Am I missing something?

ramgiteng commented 2 years ago

@diemol , No, that is actually the total script. So, there is no parallel session configuration that I am setting in the script itself.

Do I need to set the parallel session capability in the script itself in order for docker-selenium container to perform the test in parallel?

I was thinking the SE_NODE_MAX_SESSIONS=3 and SE_NODE_OVERRIDE_MAX_SESSIONS=true would take care of triggering the parallelism. Or, is this only for setting the maximum number of sessions?

diemol commented 2 years ago

If you want to run 3 sessions in parallel, you need to start those 3 sessions on the client side. Grid helps to process those requests and run them in browsers when resources are available, it does not automatically run your requests in parallel.

You could do that either by launching the script 3 times in parallel at the OS level or use a JS library with that feature, like https://mochajs.org/.

Closing as this is not a docker selenium issue.

nakodavati commented 1 year ago

i have set it up docker containers for chromenode,selenium-hub ,and my automation project "Attaching to selenium-hub, chrome, ssautomation",i am executing my dockerfile now. but when i run mvn test i am getting sessionnot created error

github-actions[bot] commented 10 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.