ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.53k stars 1.13k forks source link

There are always zombie process #1363

Closed yanxiang-wang closed 1 year ago

yanxiang-wang commented 1 year ago

When I run the uc in a docker, there are zombie processes after I called

driver.close()
driver.quit()

I tried dumb-init, but it doesn't work.

juanfrilla commented 1 year ago

@yanxiang-wang test this solution https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/649.

chrome.quit()
if hasattr(chrome, "service") and getattr(chrome.service, "process", None):
    chrome.service.process.wait(3)
os.waitpid(chrome.browser_pid, 0)

If the process hasnt been killed, wait for It to be killed.

yanxiang-wang commented 1 year ago

@yanxiang-wang test this solution #649.

chrome.quit()
if hasattr(chrome, "service") and getattr(chrome.service, "process", None):
    chrome.service.process.wait(3)
os.waitpid(chrome.browser_pid, 0)

If the process hasnt been killed, wait for It to be killed.

Thanks, it works.

MacMarde commented 1 year ago

I have the same issue but no solution is working for me. I get the error: ChildProcessError(10, 'No child processes') when trying os.waitpid(chrome.browser_pid, 0) as described above. It seems like the chrome processes get closed with chrome.quit(), but immediately there appear two new chrome background processes with a high cpu usage. The 2 new processes never get closed and are running forever consuming my CPU.

momo286 commented 1 year ago

Hello, i had the same problem, i use it in Ubuntu, and i found a solution: At the beginning of my script i run this code snippet that kills all the processes with the name "Chrome" in it:

search_string = "chrome"
command = ["pkill", "-f", search_string]
try:
    subprocess.run(command, check=True)
except subprocess.CalledProcessError:
    pass
juanfrilla commented 1 year ago

My solution was to use another alternative like playwright or ichrome, and undetected_chromedriver only in cloudflare pages, cuz its prone to errors.

MacMarde commented 1 year ago

@momo286 I am already doing this, but it does not help, because I do have multiple threads of chrome running, so I can not kill all chrome processes. I kill all chrome processes before I start my script but then it has to work without any killing of all chrome processes.

@juanfrilla How does it help? As far as I see it is not a solution for me. I do need a selenium solution with a not detectable chromedriver.

juanfrilla commented 1 year ago

@MacMarde If the page you are scraping has cloudflare then you need obligatory to use undetected-chromedriver because there arent more alternatives, but if it has not cloudflare recaptchas, then you can use playwright or ichrome. For what Im understanding, UC its only to bypass cloudflare, if you only need a browser, you can use the others i mentioned before. Your page has cloudflare?

MacMarde commented 1 year ago

@juanfrilla I do not have cloudflare recaptchas but there are way more use cases for an undetectable chromedriver.

MacMarde commented 1 year ago

I have found a workaround solution. Before I call driver.quit() I am already killing the chrome process and all child processes.

os.system(r'C:\Test\kill.bat ' + str(driver.browser_pid))
driver.quit()

This is the Batch file:

set arg1=%1
taskkill /f /T /pid %arg1%

So I get the process ID from UC and kill this process with taskkill. The parameter /T also kills all child processes. So other chrome instances should not be affected.

I think something is going wrong with UC and driver.quit(). Would still love to see it fixed.

boludoz commented 10 months ago
def kill_chrome_processes():
    try:
        subprocess.run(['taskkill', '/F', '/IM', 'chromedriver.exe', '/T'], check=True)
    except:
        pass
    try:
        subprocess.run(['taskkill', '/F', '/IM', 'chrome.exe', '/T'], check=True)
    except:
        pass

That worked for me using windows. It would be necessary to close all the processes associated only with chromedriver but for now it is enough for me.

juanfrilla commented 9 months ago

but it would be better not to kill all but kill only the correspondent spawned process per script. Because imagine you have multiple scripts or spiders running in the same machine (multiprocessing) and using undetected-chromedriver and you kill all the chrome processes, you may affect the other scripts.