ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.1k stars 1.09k forks source link

NODRIVER Memory leak #1905

Open life-live opened 1 month ago

life-live commented 1 month ago

image

ghj1976 commented 1 month ago

me too

Abdelrahman-Hekal commented 1 month ago

same here

Klimnyk commented 1 month ago

Here's my version of the function that will help you invoke the pkill command when you want to close all browser processes; this will help prevent memory leaks.

use pkill('chrome')

    def pkill(self, process_name: str) -> bool:
        try:
            if os.name == 'nt':
                os.system('taskkill /IM "' + process_name + '.exe' + '" /F')
            elif os.name == 'posix':
                os.system('pkill ' + process_name)
            else:
                return False
            killed = True
        except Exception as e:
            killed = False
        return killed
Abdelrahman-Hekal commented 1 month ago

@Klimnyk Thanks for sharing this but I don't think the leak is coming from the unterminated closed browser processes as I already implemented the below function that kills the browser processes related to the closed driver instances similar to what you suggested and still getting 50 GB of memory leak after running the code for 20 hours

    def kill_browser(self):
        try:
            process = psutil.Process(self.driver._process_pid)
            process.kill()  # Forcefully terminate the process.
        except psutil.NoSuchProcess:
            pass
        except Exception as e:
            print(f"Error terminating process {process.pid}: {e}")
Klimnyk commented 1 month ago

If the process causing the memory leak is Python, why not try killing it with pkill('python') and then restarting it?

Abdelrahman-Hekal commented 1 month ago

@Klimnyk Killing the Python process will break the script and will have to come up with a resume feature to continue where it left off when triggered again. However, what you proposed is a possible solution still.

develfe commented 1 month ago

The same problem with Python process under Centos

royial commented 3 weeks ago

any solution for this? I'm also getting 4GB of usage after 1 hour.

ZwickVitaly commented 6 days ago

after any work with certain page (or at some point of script) you need to clear mapper of browser (and cookies to be sure).

browser.connection.mapper.clear()
await borwser.cookies.clear()

connection mapper is filled up explicitly and not cleared up after interaction.

even after this solution it leaks like 125kb \ 10 mins = 750kb \ hour , but I can't locate problem. perhaps something connected to large DOM nodes, used by cdp package

Damn so good lib and so many node-like objects, that are probably not cleaned up due to python gc edge-moments

ZwickVitaly commented 4 days ago

@ultrafunkamsterdam I found out that