search

seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.46k stars 910 forks source link

UC detected on linux #2721

Closed Black-Hole1004 closed 2 months ago

Black-Hole1004 commented 2 months ago

the UC mode works fine on MacOS, but it gets detected on linux ( cloudflare )

mdmintz commented 2 months ago

Are you using a Linux machine on a shared IP Address space, such as AWS, GCP, Azure, GH Actions, etc? Those IP ranges have already been identified as non-human web traffic, so you'll get blocked for using them unless you connect to a proxy server.

I've tested on regular Linux (Ubuntu), and it's working successfully with the correct configuration.

It's working for others too, eg: https://github.com/seleniumbase/SeleniumBase/issues/2683#issuecomment-2051218740, when they use the SeleniumBase methods correctly, such as uc_click(selector) / uc_open_with_reconnect(url) etc.

Connecting to a proxy server is important when using a shared IP Address space: https://github.com/seleniumbase/SeleniumBase/issues/2063#issuecomment-1709172079

When opening tickets, make sure to include more details than that. As mentioned in the UC Mode: Video 2 tutorial (https://www.youtube.com/watch?v=2pTpBtaE7SQ), there are 3 kinds of Cloudflare CAPTCHAs, and each one must be handled differently.

In order to determine exactly why you were detected on your Linux machine, (but not on your Mac), you can run against https://pixelscan.net/ and see exactly how they're detecting you:

pixelscan

Depending on what that screen shows, that will determine the steps needed for changing your Linux configuration. A screen like that for your Linux machine will make debugging possible if I can't detect any issues with the code you used (which you would need to provide).

Black-Hole1004 commented 2 months ago

here is what it shows ( I'm not on that location btw )

Screen Shot 2024-04-27 at 4 42 07 PM
mdmintz commented 2 months ago

The No automation framework detected at the bottom is good (it means UC Mode took care of that), however, your Linux machine has a hardcoded GeoLocation, which is making you detected. This is exactly what would happen if you try to use UC Mode inside a Linux Docker container (where that is usually the case). But if you use a regular Linux Ubuntu machine (not Docker), then it should be fine.

I see your same results if I run UC Mode inside Docker: Dockerfile. But with a regular desktop Linux machine, the issue doesn't occur. (Note that UC Mode is currently not compatible with Docker, as there is some odd configuration there.)

Black-Hole1004 commented 2 months ago

Yes, because I'm running ubuntu in a cloud environment and that must be the case, so the only solution left is to use an os X simulator in linux or vm