Recently chromedriver stopped working in gipod workspace

[maximnaidenov]

Bug description

The chromedriver (https://developer.chrome.com/blog/chrome-for-testing/) that is used to start and control chrome from E2E test frameworks like WDIO stopped working in gitpod - it fails with a segmentation fault. The same chromedriver/chrome pair works perfectly fine in a Docker container in AWS VM. I detected this issue with chrome and chromedriver 127 and confirmed the problem is presented also with chromedriver 126, 125, likely all earlier. I did an ltrace for chromedriver and it shows the segmentation fault:

[1722930209.082][INFO]: Browser search. Trying... /usr/local/sbin/chrome [1722930209.131][INFO]: Browser search. Trying... /usr/local/bin/chrome [1722930209.197][INFO]: Browser search. Trying... /usr/sbin/chrome [1722930209.257][INFO]: Browser search. Trying... /usr/bin/chrome [1722930209.306][INFO]: Browser search. Found at /usr/bin/chrome [1722930210.117][INFO]: Populating Preferences file: { "alternate_error_pages": { "enabled": false }, "autofill": { "enabled": false }, "browser": { "check_default_browser": false }, "distribution": { "import_bookmarks": false, "import_history": false, "import_search_engine": false, "make_chrome_default_for_user": false, "skip_first_run_ui": true }, "dns_prefetching": { "enabled": false }, "download": { "default_directory": "./dist", "prompt_for_download": false }, "profile": { "content_settings": { "pattern_pairs": { "https://,": { "media-stream": { "audio": "Default", "video": "Default" } } } }, "default_content_setting_values": { "geolocation": 1 }, "default_content_settings": { "geolocation": 1, "mouselock": 1, "notifications": 1, "popups": 1, "ppapi-broker": 1 }, "password_manager_enabled": false }, "safebrowsing": { "enabled": false }, "search": { "suggest_enabled": false }, "translate": { "enabled": false } } [1722930210.254][INFO]: Populating Local State file: { "background_mode": { "enabled": false }, "ssl": { "rev_checking": { "enabled": false } } } [1722930210.721][INFO]: ChromeDriver supports communication with Chrome via pipes. This is more reliable and more secure. [1722930210.773][INFO]: Use the --remote-debugging-pipe Chrome switch instead of the default --remote-debugging-port to enable this communication mode. [1722930210.969][INFO]: Launching chrome: /usr/bin/chrome --allow-pre-commit-input --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-gpu --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-logging --headless --ignore-certificate-errors --log-level=0 --no-first-run --no-sandbox --no-service-autorun --password-store=basic --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir=/tmp/.org.chromium.Chromium.OuRRGh --window-size=1280,768 data:, disable_breakpoint pid=0, addr=0x5646640395b0: No such process PTRACE_SINGLESTEP: No such process 0 couldn't continue when handling pthread_mutex_trylock (0x5646640395b0) at 0xffffffffffffffff disable_breakpoint pid=8672, addr=0x5646640395b0: No such process <no return ...> +++ killed by SIGSEGV +++ gitpod@simplesoftl-recruitfron-66ukmk2p1o6:/workspace/$ [0806/074331.154002:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory [0806/074331.158467:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory [0806/074331.158554:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory [0806/074331.164590:INFO:config_dir_policy_loader.cc(118)] Skipping mandatory platform policies because no policy file was found at: /etc/opt/chrome_for_testing/policies/managed [0806/074331.164630:INFO:config_dir_policy_loader.cc(118)] Skipping recommended platform policies because no policy file was found at: /etc/opt/chrome_for_testing/policies/recommended [0806/074331.192467:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. DevTools listening on ws://127.0.0.1:44399/devtools/browser/5ebdf72f-833d-4f2b-a3f4-e79ebffc3970 [0806/074331.290784:WARNING:sandbox_linux.cc(436)] InitializeSandbox() called with multiple threads in process gpu-process.

This log shows that chromedriver is able to spawn the chrome browser process, the chrome process starts fine and even stays alive after the chromedrivier process crashes with segmentation fault. But it seems the chromedriver can't access (read id?) the child process that it spawned and crashes because it does not handle this state correctly. But the rootcause sounds like an process/container runtime insulation, exactly something that the gitpod workspace is doing.

Can you please have a look and give some hints on how to overcome it?

Steps to reproduce

Run e2e test with WDIO and local chrome.

Workspace affected

any

Expected behavior

E2E test works fine with chrome

Example repository

No response

Anything else?

No response

[axonasif]

Hi @maximnaidenov , do you have a public repo where we could reproduce this?

[maximnaidenov]

@axonasif No, I don't have a public repo but I can run whatever logs/experiments you suggest.

[axonasif]

@maximnaidenov could you kindly create a minimal repository containing a similar setup? Or if you could provide some instructions on how to replicate your setup, that would be helpful too.