seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
5.25k stars 964 forks source link

UC, Headless Not Able to Bypass Cloudfare in Linux Server #3181

Closed SanketBaraiya closed 1 week ago

SanketBaraiya commented 1 week ago

Hii team, the below piece of code works perfectly on windows but it is not working in Ubuntu Linux Server:

from seleniumbase import Driver
from bs4 import BeautifulSoup

driver = Driver(uc=True, headless=True)
driver.get("https://uk.newonnetflix.info/")
driver.sleep(25)

soup = BeautifulSoup(driver.page_source, 'html.parser')

driver.close()

I would really appreciate some help on this.

Thanks

mdmintz commented 1 week ago

On Linux, use with SB(uc=True, xvfb=True) as sb: instead of using headless mode or the Driver() format.

from seleniumbase import SB

with SB(uc=True, xvfb=True) as sb:
    sb.uc_open_with_reconnect(url)
    sb.uc_gui_click_captcha()

(Use sb.driver to access the driver)

UC Mode is detectable in regular Headless Mode, and special methods that use PyAutoGUI won't work at all in Headless Mode. (Also, the Driver() format doesn't include the xvfb option.)

Xvfb is a special virtual display for Linux machines, which lets you run a regular headed browser on a machine that does not have a GUI interface.

SanketBaraiya commented 1 week ago

Hii @mdmintz ,

I tried your recommended code but still I am not able to execute the code, Also FYI I upgraded to GUI version and i am using the below code now:

from seleniumbase import SB

with SB(uc=True) as sb:
    url="https://uk.newonnetflix.info/"
    sb.uc_open_with_reconnect(url)
    sb.uc_gui_click_captcha()

print("Done")

This is what happening it is stuck here: image

And the this error comes: image

mdmintz commented 1 week ago

For Linux machines that have a GUI, use headed=True so that it spins up a headed browser.