search

seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
5k stars 944 forks source link

Detectability Issues with Proxy Authentication Plugins in SeleniumBase During Funcaptcha Verification #3021

Closed elstrm2 closed 1 month ago

elstrm2 commented 1 month ago

I am experiencing a significant challenge with the detectability of proxy usage via SeleniumBase, particularly during automated interactions with services employing stringent security measures, such as Funcaptcha on Outlook. The core issue is related to the method of proxy authentication which uses custom Chrome extensions (plugins), and this method is being detected by such services, leading to blocking or other restrictive actions.

  1. The proxies in use are private and, on their own, do not contribute to detectability issues.
  2. My testing with Selenium Wire, which configures proxies directly without needing browser plugins, shows no detectability issues. However, SeleniumBase, which relies on plugins for proxy authentication, is detected by security mechanisms.
  3. The issue becomes particularly pronounced during interactions with Outlook's Funcaptcha, which appears to have a mechanism to detect the use of browser plugins typically employed for automation and proxy settings in SeleniumBase. Even when employing SeleniumBase’s stealth configurations, including the undetectable Chrome (uc) mode, the problem persists.
  4. All potential configurations within SeleniumBase aimed at reducing detectability have been tried, including various stealth modes and settings, but none have circumvented the detection by Funcaptcha’s security measures.

The reliance on plugins for setting proxies in SeleniumBase is notably problematic in scenarios where security measures actively seek out such extensions as part of their anti-automation protocols. This issue is particularly evident during processes like Funcaptcha verifications on platforms like Outlook, which are sensitive to any form of automation or non-standard browser behaviors.

This situation significantly limits the effectiveness of SeleniumBase for tasks requiring high levels of stealth, especially in environments where advanced security protocols against automation are in place. Detectability during critical security checks like Funcaptcha not only hinders automation tasks but also risks potential blocking and further access restrictions, making it a critical issue needing resolution or workaround.

elstrm2 commented 1 month ago

In addition to my previously sent message regarding the detectability issues with SeleniumBase's proxy authentication plugins during Funcaptcha verification on Outlook, I have thoroughly analyzed all responses provided by the author and the existing documentation. I have exhausted all recommended configurations and approaches.

Given this extensive exploration, I seek a straightforward answer: Is this issue currently resolvable with SeleniumBase, or should I anticipate changes in the library or consider using alternative solutions? Your guidance on whether immediate resolution is possible or if waiting for updates or switching libraries is advised would be greatly appreciated.

elstrm2 commented 1 month ago

In addition to my previous inquiries and based on the extensive analysis of all responses and documentation regarding SeleniumBase, I would like to propose a potential solution that could address the detectability issues associated with proxy authentication.

One viable approach could be the integration of Selenium Wire parameters directly when initializing SeleniumBase tests using the use_wire option. This would potentially allow for seamless proxy configuration without the need for detectable plugins, leveraging Selenium Wire's capabilities to handle proxies more discreetly.

Could you please confirm if incorporating Selenium Wire parameters in this manner is feasible with the current architecture of SeleniumBase, or if such an enhancement would require significant modifications to the library?

mdmintz commented 1 month ago

You mentioned:

"My testing with Selenium Wire, which configures proxies directly without needing browser plugins, shows no detectability issues."

There's a Selenium Wire integration inside SeleniumBase. See: https://github.com/seleniumbase/SeleniumBase/issues/2145

As for UC Mode (which isn't compatible with Wire Mode), that does not support FunCAPTCHA at this time. UC Mode works for Cloudflare-bypass, and also works when setting a proxy with auth.