seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.5k stars 916 forks source link

Documenting `driver`-specific methods added by SeleniumBase #2200

Open mdmintz opened 9 months ago

mdmintz commented 9 months ago

Documenting driver-specific methods added by SeleniumBase

Because there are Syntax Formats that use the driver API directly, these methods have been added directly into the documentation in the method_summary.md file: https://github.com/seleniumbase/SeleniumBase/commit/88fcf7ea50911b83d7da72346b454bb4da8adde8

# "driver"-specific methods added by SeleniumBase

driver.default_get(url)  # Because driver.get(url) works differently in UC Mode
driver.open(url)  # Like driver.get(), but allows partial URLs without protocol
driver.click(selector)
driver.click_link(link_text)
driver.click_if_visible(selector)
driver.click_active_element()
driver.send_keys(selector, text)
driver.press_keys(selector, text)
driver.type(selector, text)
driver.submit(selector)
driver.assert_element(selector)
driver.assert_element_present(selector)
driver.assert_element_not_visible(selector)
driver.assert_text(text, selector)
driver.assert_exact_text(text, selector)
driver.wait_for_element(selector)
driver.wait_for_text(text, selector)
driver.wait_for_exact_text(text, selector)
driver.wait_for_and_accept_alert()
driver.wait_for_and_dismiss_alert()
driver.is_element_present(selector)
driver.is_element_visible(selector)
driver.is_text_visible(text, selector)
driver.is_exact_text_visible(text, selector)
driver.is_attribute_present(selector, attribute)
driver.get_text(selector)
driver.js_click(selector)
driver.get_active_element_css()
driver.get_locale_code()
driver.get_origin()
driver.get_user_agent()
driver.highlight(selector)
driver.highlight_click(selector)
driver.sleep(seconds)
driver.locator(selector)
driver.get_attribute(selector, attribute)
driver.get_page_source()
driver.get_title()
driver.switch_to_frame(frame)

############

# "driver"-specific methods added by SeleniumBase for UC Mode: "--uc" / uc=True

driver.uc_open(url)
driver.uc_open_with_tab(url)
driver.uc_open_with_reconnect(url, reconnect_time=None)
driver.reconnect(timeout)
driver.uc_click(selector)
driver.uc_switch_to_frame(frame)

Here are example tests that use those direct driver methods:

boludoz commented 8 months ago
class WebDriver:
    def __init__(self):
        ua = UserAgent()

        # Create an instance of the undetected ChromeDriver in headless mode
        options = uc.ChromeOptions()

        # ua.google
        options.add_argument(f'--user-agent={ua.google}')
        options.add_argument('--ignore-certificate-errors')
        options.add_argument('--ignore-ssl-errors')

        prefs = {}
        prefs["credentials_enable_service"] = False
        prefs["profile.password_manager_enabled"] = False
        options.add_experimental_option("prefs", prefs)

        self.driver = Driver(uc=True, incognito=True, options=options)

        # self.driver = uc.Chrome(options=options)

First of all, I congratulate you on your project, it's crazy, but I would like to ask you how I would go about adding options. How i can do something like that?

mdmintz commented 8 months ago

@boludoz, thank you. Most of those are already being set automatically as needed for UC Mode.

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L997

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L772

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L791

You can change the agent with agent=AGENT (but if you change the one that SeleniumBase already sets automatically, then you could be detected). There's also chromium_arg="--some-arg,--another-arg=VALUE" for more args, comma-separated. Be sure not to set something twice that is already being set, as that could get you detected.

That assumes you are using the Driver format: https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_uc_mode.py

boludoz commented 8 months ago

You have done a great job, I really congratulate you!

bbajwa346 commented 6 months ago

@boludoz, thank you. Most of those are already being set automatically as needed for UC Mode.

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L997

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L772

https://github.com/seleniumbase/SeleniumBase/blob/f9a31a39c5544cbb5d4d290dcec9634fea14cddb/seleniumbase/core/browser_launcher.py#L791

You can change the agent with agent=AGENT (but if you change the one that SeleniumBase already sets automatically, then you could be detected). There's also chromium_arg="--some-arg,--another-arg=VALUE" for more args, comma-separated. Be sure not to set something twice that is already being set, as that could get you detected.

That assumes you are using the Driver format: https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_uc_mode.py

First and foremost, I express my gratitude for developing such an excellent framework.

I am currently encountering a small issue. I wish to utilize the Socks5 option in SeleniumBase, similar to how I would in normal uc mode. However, I am unable to find an argument option for this. Could you kindly provide a boilerplate code snippet, considering that I have a 'proxylist.txt' file and need to randomly select a proxy for each test run without manually specifying them one by one in pytest?

I've been using the following code snippet for regular mode:

options.add_argument(f"--proxy-server=socks5://{proxy_address}:{proxy_port}")

I would appreciate any guidance or assistance you can offer. Thank you.

mdmintz commented 6 months ago

@bbajwa346 https://github.com/seleniumbase/SeleniumBase/blob/31020e4ae9644596460e9606312f94bc420aff51/seleniumbase/plugins/driver_manager.py#L74 For socks5, use “socks5://“ in your “proxy” arg.

bbajwa346 commented 6 months ago

@bbajwa346

https://github.com/seleniumbase/SeleniumBase/blob/31020e4ae9644596460e9606312f94bc420aff51/seleniumbase/plugins/driver_manager.py#L74

For socks5, use “socks5://“ in your “proxy” arg.

Thank You So Much, Resolved instantly !!!

bbajwa346 commented 6 months ago

@bbajwa346

https://github.com/seleniumbase/SeleniumBase/blob/31020e4ae9644596460e9606312f94bc420aff51/seleniumbase/plugins/driver_manager.py#L74

For socks5, use “socks5://“ in your “proxy” arg.

Dear mdmintz,

one more very minor issue

from seleniumbase import Driver
driver = Driver()

driver.open("https://seleniumbase.io/coffee/")
driver.open_new_window()
# switch to new tab
driver.switch_to_window(1)
driver.open("https://google.com/")

`

I get this error driver.open_new_window() ^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'WebDriver' object has no attribute 'open_new_window'

unable to open new tab or switch between them

mdmintz commented 6 months ago

@bbajwa346 open_new_window() is not a driver-specific method. (At least not yet.) The driver already has its own methods for doing those things.

Open a new tab and switch to it: driver.switch_to.new_window("tab")

Open a new tab and stay on the current tab: driver.execute_script("window.open('');")

The standard SeleniumBase syntax formats have a lot more methods than what the driver has.