[🐛 Bug]: Selenium Manager is not thread synchronized?

marekma-gloat commented 4 weeks ago

What happened?

On the machine where I'm executing the code the correct version of the browser and driver is not preinstalled. I want to automatically download the browser and driver which matches the set version in Options(). It works perfectly if I run the code only in one thread. But if I run it in 4 threads each thread is starting to download the browser. I believe it should be downloaded only by one thread and then reused by other 3 threads. Because of this I'm getting this exception: selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited abnormally. on 3 of the threads and only 1 is successful. All tests executed after these first four are successful and I can see in the log that they're reusing the cached downloaded version. I guess it could be fixed by adding thread synchronization

How can we reproduce the issue?

Run any pytest selenium tests in parallel on machine without driver and browser preinstalled.

Relevant log output

DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:90 Selenium Manager binary found at: /usr/local/lib/python3.9/site-packages/selenium/webdriver/common/linux/selenium-manager
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:103 Executing process: /usr/local/lib/python3.9/site-packages/selenium/webdriver/common/linux/selenium-manager --browser chrome --browser-version 125 --debug --language-binding python --output json
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Sending stats to Plausible: Props { browser: "chrome", browser_version: "125", os: "linux", arch: "x86_64", lang: "python", selenium_version: "4.21" }
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 chromedriver not found in PATH
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 chrome detected at /usr/bin/google-chrome
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Running command: /usr/bin/google-chrome --version
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Output: "Google Chrome 110.0.5481.100 "
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Detected browser: chrome 110.0.5481.100
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Discovered chrome version (110) different to specified browser version (125)
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Discovering versions from https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Required browser: chrome 125.0.6422.141
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Downloading chrome 125.0.6422.141 from https://storage.googleapis.com/chrome-for-testing-public/125.0.6422.141/linux64/chrome-linux64.zip
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 chrome 125.0.6422.141 is available at /root/.cache/selenium/chrome/linux64/125.0.6422.141/chrome
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Discovering versions from https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Required driver: chromedriver 125.0.6422.141
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Downloading chromedriver 125.0.6422.141 from https://storage.googleapis.com/chrome-for-testing-public/125.0.6422.141/linux64/chromedriver-linux64.zip
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Driver path: /root/.cache/selenium/chromedriver/linux64/125.0.6422.141/chromedriver
DEBUG    selenium.webdriver.common.selenium_manager:selenium_manager.py:129 Browser path: /root/.cache/selenium/chrome/linux64/125.0.6422.141/chrome
DEBUG    selenium.webdriver.common.service:service.py:219 Started executable: `/root/.cache/selenium/chromedriver/linux64/125.0.6422.141/chromedriver` in a child process with pid: 225 using 0 to output -3
DEBUG    selenium.webdriver.remote.remote_connection:remote_connection.py:301 POST http://localhost:37277/session {'capabilities': {'firstMatch': [{}], 'alwaysMatch': {'browserName': 'chrome', 'pageLoadStrategy': <PageLoadStrategy.normal: 'normal'>, 'goog:loggingPrefs': {'performance': 'ALL'}, 'browserVersion': None, 'goog:chromeOptions': {'prefs': {'profile.default_content_settings.popups': 0, 'download.default_directory': '/home/jenkins/agent/workspace/automation_e2e/Downloads'}, 'extensions': [], 'binary': '/root/.cache/selenium/chrome/linux64/125.0.6422.141/chrome', 'args': ['--window-size=1920,1080', '--no-sandbox', '--force-device-scale-factor=1', '--remote-debugging-pipe', '--user-data-dir=/tmp/selenium_user_data_7ddbiuwl', '--headless=new', '--disable-dev-shm-usage']}}}}
DEBUG    urllib3.connectionpool:connectionpool.py:227 Starting new HTTP connection (1): localhost:37277
DEBUG    urllib3.connectionpool:connectionpool.py:452 http://localhost:37277 "POST /session HTTP/1.1" 500 960
DEBUG    selenium.webdriver.remote.remote_connection:remote_connection.py:330 Remote response: status=500 | data={"value":{"error":"session not created","message":"session not created: Chrome failed to start: exited abnormally.\n  (timeout: Timed out receiving message from renderer: 60.000)\n  (The process started from chrome location /root/.cache/selenium/chrome/linux64/125.0.6422.141/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)","stacktrace":"#0 0x559e5172ee3a \u003Cunknown>\n#1 0x559e5141845c \u003Cunknown>\n#2 0x559e5144d6f8 \u003Cunknown>\n#3 0x559e5144963b \u003Cunknown>\n#4 0x559e51493b19 \u003Cunknown>\n#5 0x559e51487253 \u003Cunknown>\n#6 0x559e514571c7 \u003Cunknown>\n#7 0x559e51457b3e \u003Cunknown>\n#8 0x559e516f527b \u003Cunknown>\n#9 0x559e516f9327 \u003Cunknown>\n#10 0x559e516e1dae \u003Cunknown>\n#11 0x559e516f9df2 \u003Cunknown>\n#12 0x559e516c674f \u003Cunknown>\n#13 0x559e5171e128 \u003Cunknown>\n#14 0x559e5171e2fb \u003Cunknown>\n#15 0x559e5172df6c \u003Cunknown>\n#16 0x7fafa6d3eea7 start_thread\n"}} | headers=HTTPHeaderDict({'Content-Length': '960', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
DEBUG    selenium.webdriver.remote.remote_connection:remote_connection.py:357 Finished Request

Operating System

Debian

Selenium version

Python 4.21.0

What are the browser(s) and version(s) where you see this issue?

Chrome 125

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 125.0.6422.141

Are you using Selenium Grid?

-

github-actions[bot] commented 4 weeks ago

@marekma-gloat, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

marekma-gloat commented 3 weeks ago

Never mind, I guess it's because I'm using pytest-xdist, which seems to be running tests in multiple processes, not threads. I solved it by using FileLock for browser and driver download/creation, maybe it would be nice if it would be built in Selenium for python...

SeleniumHQ / selenium