wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

selenium-wire gets detected immediately #566

Closed anhhung4u closed 2 years ago

anhhung4u commented 2 years ago

I'm trying to use selenium-wire 4.6.4 to crawl some data. When I visited these sites https://www.oddschecker.com/horse-racing and https://nowsecure.nl (for testing bot detection), it got detected immediately.

I've tested with undetected-chromedriver 3.1.2 and it passed cloudflare on both 2 sites above. I've tested with selenium-wire and got detected by both 2 sites above. I've tested with selenium-wire.undetected_chromedriver and got detected by both 2 sites above.

I'm using Selenium-wire 4.6.4 and undetected-chromedriver 3.1.2

Here is my code for testing:

import seleniumwire.undetected_chromedriver as UCD

import time

def main():
  options = UCD.ChromeOptions()
  options.add_argument(f'--user-data-dir=D:/Chrome-profile/profile0')

  driver = UCD.Chrome(options=options, seleniumwire_options={})

  driver.get('https://www.oddschecker.com/horse-racing')  # test bot detection
  time.sleep(60)

if __name__ == '__main__':
  main()

Does anyone know how to solve this problem? Thanks in advance.

wkeeling commented 2 years ago

Thank you for raising this. Yes unfortunately it's a known issue. The current working assumption is that Cloud Flare is able to detect Selenium Wire via it's TLS handshake (TLS fingerprint). Selenium Wire presents a different fingerprint to a browser because it decrypts HTTPS traffic in order to intercept requests. We're tracking the issue in #215 . Unfortunately there's no solution at the current time.

adirzoari commented 1 year ago

@anhhung4u @wkeeling I have the same issue. is there any solution?