seleniumbase / SeleniumBase

📊 Blazing fast Python framework for web crawling, scraping, testing, and reporting. Supports pytest. Stealth options: UC Mode and CDP Mode. Multiple tools and integrations.
https://seleniumbase.io
MIT License
5.38k stars 979 forks source link

Is there any method in combination with selenium-stealth ? #942

Closed thanhvodad closed 3 years ago

thanhvodad commented 3 years ago

https://pypi.org/project/selenium-stealth/ It is a library that combines with selenium to stealth over google, is there any way to integrate it into seleniumbase, it's very good

mdmintz commented 3 years ago

By the looks of it, you can integrate it directly into the custom setUp() method of your tests like this after you pip install it:

from seleniumbase import BaseCase
from selenium_stealth import stealth

class BaseTestCase(BaseCase):
    def setUp(self):
        super().setUp()
        stealth(self.driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
        )

It's that simple. No extra code is required on the SeleniumBase side.

thanhvodad commented 3 years ago

By the looks of it, you can integrate it directly into the custom setUp() method of your tests like this after you pip install it:

from seleniumbase import BaseCase
from selenium_stealth import stealth

class BaseTestCase(BaseCase):
    def setUp(self):
        super().setUp()
        stealth(self.driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
        )

It's that simple. No extra code is required on the SeleniumBase side.

How can I choose my custom chromedriver file?.

mdmintz commented 3 years ago

@thanhvodad The chromedriver used is the one that's on your system path. So if you want to use a specific chromedriver, make sure that it's the only one on the path. SeleniumBase adds the seleniumbase/drivers folder to your path when you run tests, so any driver installed there via a command such as sbase install chromedriver latest would be used before any other one that's found elsewhere. Maybe do a search for chromedriver to see all occurrences of it on your computer in case you're not sure which ones exist.

thanhvodad commented 3 years ago
def setUp(self):
    if self.browser == "chrome":
        super().setUp()
        stealth(
            self.driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
        )

object has no attribute 'browser' self.browser not in setup ?

mdmintz commented 3 years ago

@thanhvodad You can call the self commands in your setUp() method only after you've run super().setUp(), which is on the very next line in your example.

thanhvodad commented 3 years ago

what will be the correct command, when i want to check if it's browe.chorme, if yes i will use stealth

mdmintz commented 3 years ago

@thanhvodad

def setUp(self):
    super().setUp()
    if self.browser == "chrome":
        stealth(
            self.driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
        )

(Make sure you call super().setUp() before you call any self commands.)

thanhvodad commented 3 years ago

I tried, yes seleniumbase has been caught by google automatically, I login google it will say :(

mdmintz commented 3 years ago

@thanhvodad If you're looking to avoid Selenium detection, you should see https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver/52108199#52108199 and the solution for that here: https://stackoverflow.com/a/52108199

In summary, you'll have to modify chromedriver so that websites don't know how to detect it anymore. Here's how you can do that with vim: vim /path/to/chromedriver

After running the line above, you'll probably see a bunch of gibberish. Do the following:

  1. Replace all instances of cdc_ with dog_ by typing :%s/cdc_/dog_/g.
    • dog_ is just an example. You can choose anything as long as it has the same amount of characters as the search string (e.g., cdc_), otherwise the chromedriver will fail.
  2. To save the changes and quit, type :wq! and press return.
    • If you need to quit without saving changes, type :q! and press return.