seleniumbase / SeleniumBase

📊 Blazing fast Python framework for web crawling, scraping, testing, and reporting. Supports pytest. Stealth abilities: UC Mode and CDP Mode.
https://seleniumbase.io
MIT License
5.43k stars 982 forks source link

uc=True Leaving Zombie Processes #3173

Closed felipehertzer closed 1 month ago

felipehertzer commented 1 month ago

Hey @mdmintz,

I'm having a problem when using uc=True on Mac and Linux. It is leaving some zombie processes that never finish.

Also, if I try to quit the driver with uc=True, it runs forever:

from seleniumbase import SB
with SB(uc=True) as sb:
    sb.uc_open_with_reconnect("https://www.google.com", 4)
    sb.driver.quit()

However, if I set uc=False, the script runs and quits fine, and no zombie processes are left behind.

Expected Behavior:

The script should run and exit cleanly when using uc=True without leaving any zombie processes.

Observed Behavior:

When uc=True is set, the script hangs on driver.quit() and creates zombie processes that persist.

Additional Information:

This issue occurs on both Mac and Linux (Alpine and Ubuntu) environments. Here is a screenshot of the process table showing the zombie processes:

image

Environment:

Steps to Reproduce:

  1. Run the provided script with uc=True.
  2. Observe the process table after sb.driver.quit() hangs.

Let me know if you need more details or if I can assist further.

Thanks!

mdmintz commented 1 month ago

For the context manager formats (where there's a with block) the driver is quit in a special way automatically at the end of the with block (at the end of the indented block of code that starts with with SB(uc=True) as sb:). From those formats, the driver shouldn't be quit manually before the end of the with block. (You were quitting the driver before the with block ended.)

Also, you appear to be using Chromium instead of regular Chrome (by the look of your screenshot). You need to use a regular Chrome browser for UC Mode (on Linux, it'll be either google-chrome-stable or google-chrome, not chromium).

Also see https://github.com/seleniumbase/SeleniumBase/issues/2953, which may have relevant info. (You can use psutil - https://pypi.org/project/psutil/ to terminate any processes left behind if you're still having issues.)

felipehertzer commented 1 month ago

I understand the context. The quit statement in my code was not intentional but rather a desperate attempt to terminate the process by force. I am using Chromium due to the use of an arch64 processor. I have also attempted to use the Brave browser but encountered the same issue. Furthermore, I tried terminating the process manually, but it was unsuccessful. I will try again.

felipehertzer commented 1 month ago

For the record, I'm using selenium inside a docker container and to fix the zombie issue I had to deploy it with the following options

    ipc: host
    init: true

Using --ipc=host is also recommended when using Chromium. Without it Chromium can run out of memory and crash. Using --init Docker flag or dumb-init is recommended to avoid special treatment for processes with PID=1. This is a common reason for zombie processes.