ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.08k stars 1.09k forks source link

[NODRIVER] VERY IMPORTANT MEMORY LEAK FIX #1922

Closed zxsleebu closed 8 hours ago

zxsleebu commented 3 weeks ago

@ultrafunkamsterdam i have found a fix for the memory leak everybody was experiencing. here is the link to my comment in another issue. i hope you'll take a look at this https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1851#issuecomment-2167380451 only 1 missing line of code causes memory leak

jwwq commented 3 weeks ago

Thank you so much, man, for that fix! I'm really tired of the fact that parsing a few hundred pages completely drains memory. Today I've narrowed down the leak to tab.query_selector (and hence to CDP call that returns items), but tired as hell, used a dirty hack with running nodriver in separate process to completely clear memory, did a reprex for the bug report, and without much hope, I entered "nodriver memory leak" and found your solution that you posted just two days ago! What a happy coincidence!

God bless you!

jwwq commented 3 weeks ago

I can confirm that the fix works like a charm!

image

zxsleebu commented 3 weeks ago

Thank you so much, man, for that fix! I'm really tired of the fact that parsing a few hundred pages completely drains memory. Today I've narrowed down the leak to tab.query_selector (and hence to CDP call that returns items), but tired as hell, used a dirty hack with running nodriver in separate process to completely clear memory, did a reprex for the bug report, and without much hope, I entered "nodriver memory leak" and found your solution that you posted just two days ago! What a happy coincidence!

God bless you!

I'm glad I helped you and I really hope that the lead developer pushes this fix ASAP. thank you for the kind words!

FDDQA commented 4 days ago

Unfortunately, this fix doesn't work. I also tried using a temporary file and adding a line to the library file - each time a browser.get(url) consumes about 100 KB. Also didn't help: page.close, page del, gc.collect(), etc.

ultrafunkamsterdam commented 9 hours ago

this is implemented. thanks @zxsleebu

zxsleebu commented 8 hours ago

this is implemented. thanks @zxsleebu

glad that you approved this fix. thanks for the library!