search

SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.49k stars 8.15k forks source link

[🐛 Bug]: unable to run selenium python in a docker container #13376

Closed alfonsocanor closed 9 months ago

alfonsocanor commented 9 months ago

What happened?

Python selenium running from a docker container results in timeout: Timed out receiving message from renderer: 60.000. Locally in MacOs (M2) works as expected.

How can we reproduce the issue?

- Create a python main.py
- configure this options chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--remote-debugging-pipe')
The last option suggested from https://github.com/SeleniumHQ/selenium/issues/13185
- Instantiate the driver using:     driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
- Exception: timeout: Timed out receiving message from renderer: 60.000

Relevant log output

File "/app/main.py", line 99, in <module>
    reschedule_appointments()
  File "/app/main.py", line 79, in reschedule_appointments
    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 209, in __init__
    self.start_session(capabilities)
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 293, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 348, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: crashed.
  (timeout: Timed out receiving message from renderer: 60.000)
  (The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

Operating System

MacOs + Docker

Selenium version

webdriver-manager==4.0.1

What are the browser(s) and version(s) where you see this issue?

120

What are the browser driver(s) and version(s) where you see this issue?

120

Are you using Selenium Grid?

NA

github-actions[bot] commented 9 months ago

@alfonsocanor, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

titusfortner commented 9 months ago

We do not write or maintain webdriver manager. Selenium has added driver and browser management functionality natively in our library, though. If you use the latest selenium without webdriver manager and still have an issue, we can help, otherwise

alfonsocanor commented 9 months ago

I remove the webdriver manager dependency. Here's how it looks now same exception:

titusfortner commented 9 months ago

Run it with logging turned on:

https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

alfonsocanor commented 9 months ago

Information from logs:

Started executable: /usr/local/bin/chromedriver in a child process with pid: 94 using 0 to output -3 POST http://localhost:55197/session Remote response: status=500 | data={"value":{"error":"session not created","message":"session not created: Chrome failed to start: > Finished Request

titusfortner commented 9 months ago

Hmm, I think our docs are missing the basicConfig line. Should look like this:

import logging
from selenium import webdriver

logging.basicConfig(level=logging.WARN)
logging.getLogger('selenium').setLevel(logging.DEBUG)

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--remote-debugging-pipe')
driver = webdriver.Chrome(chrome_options)
alfonsocanor commented 9 months ago

DEBUG:selenium.webdriver.common.selenium_manager:Selenium Manager binary found at: /usr/local/lib/python3.12/site-packages/selenium/webdriver/common/linux/selenium-manager DEBUG:selenium.webdriver.common.selenium_manager:Executing process: /usr/local/lib/python3.12/site-packages/selenium/webdriver/common/linux/selenium-manager --browser chrome --debug --output json DEBUG:selenium.webdriver.common.selenium_manager:Found chromedriver 120.0.6099.109 in PATH: /usr/local/bin/chromedriver DEBUG:selenium.webdriver.common.selenium_manager:chrome detected at /usr/bin/google-chrome DEBUG:selenium.webdriver.common.selenium_manager:Running command: /usr/bin/google-chrome --version DEBUG:selenium.webdriver.common.selenium_manager:Output: "Google Chrome 120.0.6099.129 " DEBUG:selenium.webdriver.common.selenium_manager:Detected browser: chrome 120.0.6099.129 DEBUG:selenium.webdriver.common.selenium_manager:Required driver: chromedriver 120.0.6099.109 DEBUG:selenium.webdriver.common.selenium_manager:Driver path: /usr/local/bin/chromedriver DEBUG:selenium.webdriver.common.selenium_manager:Browser path: /usr/bin/google-chrome DEBUG:selenium.webdriver.common.selenium_manager:Using driver at: /usr/local/bin/chromedriver DEBUG:selenium.webdriver.common.service:Started executable: /usr/local/bin/chromedriver in a child process with pid: 379 using 0 to output -3 DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:49415/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "browserVersion": null, "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/google-chrome", "args": ["--no-sandbox", "--headless", "--disable-dev-shm-usage", "--remote-debugging-pipe"]}}}} DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=500 | data={"value":{"error":"session not created","message":"session not created: Chrome failed to start: crashed.\n (timeout: Timed out receiving message from renderer: 60.000)\n (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)","stacktrace":"#0 0x0040007a5f83 \u003Cunknown>\n#1 0x00400045ecf7 \u003Cunknown>\n#2 0x00400049660e \u003Cunknown>\n#3 0x00400049326e \u003Cunknown>\n#4 0x0040004e380c \u003Cunknown>\n#5 0x0040004d7e53 \u003Cunknown>\n#6 0x00400049fdd4 \u003Cunknown>\n#7 0x0040004a11de \u003Cunknown>\n#8 0x00400076a531 \u003Cunknown>\n#9 0x00400076e455 \u003Cunknown>\n#10 0x004000756f55 \u003Cunknown>\n#11 0x00400076f0ef \u003Cunknown>\n#12 0x00400073a99f \u003Cunknown>\n#13 0x004000793008 \u003Cunknown>\n#14 0x0040007931d7 \u003Cunknown>\n#15 0x0040007a5124 \u003Cunknown>\n#16 0x004002ca0044 \u003Cunknown>\n"}} | headers=HTTPHeaderDict({'Content-Length': '916', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) DEBUG:selenium.webdriver.remote.remote_connection:Finished Request Traceback (most recent call last): File "/app/main.py", line 105, in reschedule_appointments() File "/app/main.py", line 85, in reschedule_appointments driver = webdriver.Chrome(options=chrome_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in init super().init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in init super().init(command_executor=executor, options=options) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 209, in init self.start_session(capabilities) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 293, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 348, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: crashed. (timeout: Timed out receiving message from renderer: 60.000) (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace:

0 0x0040007a5f83

1 0x00400045ecf7

2 0x00400049660e

3 0x00400049326e

4 0x0040004e380c

5 0x0040004d7e53

6 0x00400049fdd4

7 0x0040004a11de

8 0x00400076a531

9 0x00400076e455

10 0x004000756f55

11 0x00400076f0ef

12 0x00400073a99f

13 0x004000793008

14 0x0040007931d7

15 0x0040007a5124

16 0x004002ca0044

titusfortner commented 9 months ago

Hmm, now add driver logging...

service = webdriver.ChromeService(service_args=['--log-level=DEBUG'], log_output=subprocess.STDOUT)
driver = webdriver.Chrome(chrome_options, service)
alfonsocanor commented 9 months ago

new logs: (Updated with '--whitelisted-ips=')

DEBUG:selenium.webdriver.common.selenium_manager:Selenium Manager binary found at: /usr/local/lib/python3.12/site-packages/selenium/webdriver/common/linux/selenium-manager DEBUG:selenium.webdriver.common.selenium_manager:Executing process: /usr/local/lib/python3.12/site-packages/selenium/webdriver/common/linux/selenium-manager --browser chrome --debug --output json DEBUG:selenium.webdriver.common.selenium_manager:chromedriver not found in PATH DEBUG:selenium.webdriver.common.selenium_manager:chrome detected at /usr/bin/google-chrome DEBUG:selenium.webdriver.common.selenium_manager:Running command: /usr/bin/google-chrome --version DEBUG:selenium.webdriver.common.selenium_manager:Output: "Google Chrome 120.0.6099.129 " DEBUG:selenium.webdriver.common.selenium_manager:Detected browser: chrome 120.0.6099.129 DEBUG:selenium.webdriver.common.selenium_manager:Required driver: chromedriver 120.0.6099.109 DEBUG:selenium.webdriver.common.selenium_manager:chromedriver 120.0.6099.109 already in the cache DEBUG:selenium.webdriver.common.selenium_manager:Driver path: /root/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver DEBUG:selenium.webdriver.common.selenium_manager:Browser path: /usr/bin/google-chrome DEBUG:selenium.webdriver.common.selenium_manager:Using driver at: /root/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver DEBUG:selenium.webdriver.common.service:Started executable: /root/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver in a child process with pid: 730 using 0 to output None Starting ChromeDriver 120.0.6099.109 (3419140ab665596f21b385ce136419fde0924272-refs/branch-heads/6099@{#1483}) on port 45571 All remote connections are allowed. Use an allowlist instead! Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe. ChromeDriver was started successfully. DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:45571/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "browserVersion": null, "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/google-chrome", "args": ["--no-sandbox", "--headless", "--disable-dev-shm-usage", "--remote-debugging-pipe"]}}}} [1704136504.496][INFO]: [0866b3ec234e4b853dfcebc845dd5f92] COMMAND InitSession { "capabilities": { "alwaysMatch": { "browserName": "chrome", "browserVersion": null, "goog:chromeOptions": { "args": [ "--no-sandbox", "--headless", "--disable-dev-shm-usage", "--remote-debugging-pipe" ], "binary": "/usr/bin/google-chrome", "extensions": [ ] }, "pageLoadStrategy": "normal" }, "firstMatch": [ { } ] } } [1704136504.509][INFO]: Populating Preferences file: { "alternate_error_pages": { "enabled": false }, "autofill": { "enabled": false }, "browser": { "check_default_browser": false }, "distribution": { "import_bookmarks": false, "import_history": false, "import_search_engine": false, "make_chrome_default_for_user": false, "skip_first_run_ui": true }, "dns_prefetching": { "enabled": false }, "profile": { "content_settings": { "pattern_pairs": { "https://,": { "media-stream": { "audio": "Default", "video": "Default" } } } }, "default_content_setting_values": { "geolocation": 1 }, "default_content_settings": { "geolocation": 1, "mouselock": 1, "notifications": 1, "popups": 1, "ppapi-broker": 1 }, "password_manager_enabled": false }, "safebrowsing": { "enabled": false }, "search": { "suggest_enabled": false }, "translate": { "enabled": false } } [1704136504.509][INFO]: Populating Local State file: { "background_mode": { "enabled": false }, "ssl": { "rev_checking": { "enabled": false } } } [1704136504.512][INFO]: Launching chrome: /usr/bin/google-chrome --allow-pre-commit-input --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-logging --headless --log-level=0 --no-first-run --no-sandbox --no-service-autorun --password-store=basic --remote-debugging-pipe --test-type=webdriver --use-mock-keychain --user-data-dir=/tmp/.org.chromium.Chromium.GizIJX data:, [1704136504.525][DEBUG]: DevTools WebSocket Command: Browser.getVersion (id=1) (session_id=) browser { } [1704136564.526][SEVERE]: Timed out receiving message from renderer: 60.000 [1704136565.019][DEBUG]: Failed to obtain browser info: timeout: Timed out receiving message from renderer: 60.000 [1704136565.021][DEBUG]: Connection terminated while reading from pipe [1704136565.418][INFO]: [0866b3ec234e4b853dfcebc845dd5f92] RESPONSE InitSession ERROR session not created: Chrome failed to start: crashed. (timeout: Timed out receiving message from renderer: 60.000) (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) [1704136565.420][DEBUG]: Log type 'driver' lost 1 entries on destruction [1704136565.421][DEBUG]: Log type 'browser' lost 0 entries on destruction DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=500 | data={"value":{"error":"session not created","message":"session not created: Chrome failed to start: crashed.\n (timeout: Timed out receiving message from renderer: 60.000)\n (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)","stacktrace":"#0 0x0040007a5f83 \u003Cunknown>\n#1 0x00400045ecf7 \u003Cunknown>\n#2 0x00400049660e \u003Cunknown>\n#3 0x00400049326e \u003Cunknown>\n#4 0x0040004e380c \u003Cunknown>\n#5 0x0040004d7e53 \u003Cunknown>\n#6 0x00400049fdd4 \u003Cunknown>\n#7 0x0040004a11de \u003Cunknown>\n#8 0x00400076a531 \u003Cunknown>\n#9 0x00400076e455 \u003Cunknown>\n#10 0x004000756f55 \u003Cunknown>\n#11 0x00400076f0ef \u003Cunknown>\n#12 0x00400073a99f \u003Cunknown>\n#13 0x004000793008 \u003Cunknown>\n#14 0x0040007931d7 \u003Cunknown>\n#15 0x0040007a5124 \u003Cunknown>\n#16 0x004002ca0044 \u003Cunknown>\n"}} | headers=HTTPHeaderDict({'Content-Length': '916', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}) DEBUG:selenium.webdriver.remote.remote_connection:Finished Request Traceback (most recent call last): File "/app/main.py", line 105, in test_function() File "/app/main.py", line 85, in test_function driver = webdriver.Chrome(service=service, options=chrome_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in init super().init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in init super().init(command_executor=executor, options=options) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 209, in init self.start_session(capabilities) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 293, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 348, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: crashed. (timeout: Timed out receiving message from renderer: 60.000) (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

titusfortner commented 9 months ago

Oh. Linux on M2. This makes sense now.

Selenium does not automatically support Linux on ARM computers (e.g., M2). (Google does not provide browsers or drivers for this architecture, see https://github.com/GoogleChromeLabs/chrome-for-testing/issues/1)

Documentation here — https://www.selenium.dev/documentation/selenium_manager/#alternative-architectures

alfonsocanor commented 9 months ago

Thanks for the time spent and the information. I found that Firefox supports this execution, in my scenario, I don't need to explicitly use Chrome but selenium.

Here's an official docker image and additional information: https://github.com/seleniumhq-community/docker-seleniarm .Running as seluser worked as expected.

github-actions[bot] commented 8 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.