seleniumbase / SeleniumBase

📊 Blazing fast Python framework for web crawling, scraping, testing, and reporting. Supports pytest. Stealth abilities: UC Mode and CDP Mode.
https://seleniumbase.io
MIT License
5.45k stars 985 forks source link

Proxy Causing Browser Crash #3012

Closed ttraxxrepo closed 3 months ago

ttraxxrepo commented 3 months ago

Screenshot 2024-08-11 at 1 22 40 AM

When using proxy, I get the above message in the browser... The stability definitely seems to be getting effected as my program works fine when proxy is removed.

with SB(uc=True, test=True, slow=True, demo_sleep=3, proxy=<redacted>) as sb:

Is there anyway to keep stability in-tact and use proxy with the SB context manager?

the portion of the test is crashing right when the recaptcha checkbox is selected and the images to select pop-up

mdmintz commented 3 months ago

The warning message, "you are using an unsupported command-line flag ignore-certificate-errors. Stability and security will suffer", is harmless when shown. There's a command-line option to hide it, but using that option makes UC Mode detectable. Therefore, having that message appear is the better way to go.

If something crashes, I need to see the full stack trace in order to debug. Might also be an issue with your proxy.

UC Mode does not currently bypass Google reCAPTCHA. Use UC Mode for bypassing Cloudflare Turnstile.

ttraxxrepo commented 3 months ago

Thanks @mdmintz and apologies I realize I did not provide much info to go on, was initially thinking there may be a known issue since that message that popped up called out potential issues with stability.

I don’t believe the issue is with the proxy itself, as when I turn off the proxy in SB, but use WireGuard client on my Mac, everything works. I am using an AirVPN connection as my proxy.

Im not in front of PC right now but I don’t recall anything in the venv console output being very helpful to paste here - what’s the best way to get the full stack trace? I don’t think it errored out as the chrome window just froze, maybe I need to just wait longer though as I was the one force closing it after 30 seconds or so.

Also, I am aware SeleniumBase does not support Recaptcha, I am using my own seperate code to solve them which I also don’t think is the issue here since it works when not using proxy in SB and also when the host (osx) WireGuard client is connected

mdmintz commented 3 months ago

I'm not familiar with WireGuard or AirVPN. I do regularly test with my own proxies, and haven't encountered that issue. Try waiting longer to see if there is an error message with a stack trace (maybe wait a full 5 minutes). Without a stack trace, I wouldn't even know where to start looking for your issue. Maybe try raising the timeout values, etc. Also helpful would be the line of code you ran before the issue happened.

ttraxxrepo commented 3 months ago

I’ll keep playing with it - how can I do no-sandbox and disable-gpu with SB()?

I don’t think it’s actually captcha related as I got past it now, so think it’s resource related, hence trying to play with some of these chrome options… are there specific options that are getting enabled additionally when using proxy?

mdmintz commented 3 months ago

Explore SeleniumBase/seleniumbase/core/browser_launcher.py to see what's being set.