search

seleniumbase / SeleniumBase

📊 Blazing fast Python framework for web crawling, scraping, testing, and reporting. Supports pytest. Stealth abilities: UC Mode and CDP Mode.
https://seleniumbase.io
MIT License
5.44k stars 983 forks source link

driver.uc_gui_click_captcha() Broken #3227

Closed p-rk closed 1 month ago

p-rk commented 1 month ago

Since chrome version - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36.

driver.uc_gui_click_captcha() is broken. It is not clicking on the cloudflare turnstile captcha when accessing the website.

Tried below it was working earlier, but suddenly after uc driver update it stopped working.

Screenshot 2024-10-27 at 4 00 53 PM
mdmintz commented 1 month ago

Which version of SeleniumBase are you using?

mdmintz commented 1 month ago

Cannot reproduce your issue. The following code is bypassing the CAPTCHA for me on the latest version of SeleniumBase (4.32.3):

from seleniumbase import Driver

driver = Driver(uc=True)
driver.uc_open_with_reconnect("https://flash-files.com/")
driver.uc_gui_click_captcha()
driver.quit()
p-rk commented 1 month ago

I'm currently using seleniumbase v4.30.8

mdmintz commented 1 month ago

I'm currently using seleniumbase v4.30.8

Upgrade to 4.32.3 - (or newer if available).

p-rk commented 1 month ago

I tried upgrading it to the latest version and seems like I'm still having issues. Someone also reported yesterday the same issue.

Screenshot 2024-10-27 at 5 46 06 PM
mdmintz commented 1 month ago

What happens when you run a script such as this?

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://flash-files.com/"
    sb.uc_open_with_reconnect(url, 4)
    sb.uc_gui_click_captcha()
    sb.sleep(5)

For me, it loads the page. Then, after locating the CAPTCHA checkbox with Selenium, it reloads the page again without Selenium so that the checkbox can be clicked with PyAutoGUI. Then the CAPTCHA is bypassed.

p-rk commented 1 month ago

Thanks for quick replies really appreciate. Here is the video recording for me it opens up and then closes after a while without even clicking on captcha but the log says passed. I tried to extract cookies using sb.get_cookies() but it is empty also. So basically it is not clicking on the captcha at all. Kindly check below recording.

https://github.com/user-attachments/assets/445e494f-f632-4250-8c25-3f132a301c54

mdmintz commented 1 month ago

It looks like it didn't even attempt the page refresh that was expected. It could mean that you don't have PyAutoGUI installed, or that you have multiple versions of Python (or multiple virtual environments) installed on your system (and that the env that actually ran the script wasn't the latest one).

Check that py -m seleniumbase --version shows the same version as seleniumbase --version. Check that PyAutoGUI is installed. If both True, then need to go into the debugger (put in a breakpoint() before the sb.uc_gui_click_captcha() line. Then type s and press Enter to step into methods. n and Enter to go through lines in the same method. That will help debug the issue.

Also from your script (or breakpoint), you can use print(sb.version_info) to see if the version being used is correct.

p-rk commented 1 month ago

checked pyAutoGUI already installed, it used to work with old UC v129 without any issues earlier. But after UC updated to v130 it started behaving like this. I only have one python version on system. Is there anyway to downgrade to UC to 129.0 ?

Checked with commands you shared and they both log seleniumbase same version v4.32.3.

Screenshot 2024-10-27 at 6 30 34 PM
mdmintz commented 1 month ago

I'm looking into it, but so far can't reproduce this. (Probably caused by the changed in Chrome 130). If you know a bit about debugging and are able to see where code was skipped in browser_launcher.py, that would be helpful in identifying this issue.

mortalis13 commented 4 weeks ago

Not sure if it helps, but I added print(sb.driver.page_source) after the sleep and it shows this (for the sample script provided earlier https://github.com/seleniumbase/SeleniumBase/issues/3227#issuecomment-2439994923):

<html><head></head><body>
  <script src="audio.js"></script><audio></audio>

</body></html>

Didn't check what it printed when it was working, but I think it should be the HTML of the page with captcha.
Methods like is_text_visible() don't work either, for that reason I suppose.

print(sb.version_info) -> (4, 32, 3)

Windows 10 Python 3.11.9 selenium 4.25.0 seleniumbase 4.32.3

Tell me if you need the full pip list.

mortalis13 commented 4 weeks ago

Also, that empty HTML is printed after uc_open_with_reconnect and after uc_gui_click_captcha.
And before uc_open_with_reconnect it's a bunch of SVG, with <title>New Tab</title> so it's a Chrome new tab I guess.

mortalis13 commented 4 weeks ago

When debugging I see it reaches the line that checks the page source but jumps to early return before installing PyAutoGUI (with the page source as in the previous comment) image

mdmintz commented 4 weeks ago

That helps... Thank you. The real bug is with driver.connect() and driver.reconnect(). Instead of reconnecting to the correct tab, they reconnect to an invisible built-in Chrome extension tab. I think I can fix it for the next release.

mdmintz commented 4 weeks ago

I created https://github.com/seleniumbase/SeleniumBase/issues/3231 for the real issue (with driver.connect() and driver.reconnect()).

Other methods expect the driver to be connected to the browser (and on the correct tab) to work properly. The changes in Chrome 130 were causing an invisible Chrome extension tab to appear. (Seen with driver.window_handles.)