wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

wait_for_request times out when using proxy #463

Closed calum-mcg closed 2 years ago

calum-mcg commented 2 years ago

Recently changed from selenium to selenium-wire and having difficulty using wait_for_request when a proxy is set. The wait_for_request times out when I use the proxy. I've tested without the proxy and it works brilliantly. Any help would be greatly appreciated!

This is how i'm setting up the driver:

from seleniumwire import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--ignore-certificate-errors-spki-list')

desired_capabilities = webdriver.DesiredCapabilities.CHROME.copy()
desired_capabilities['proxy'] = {
    'httpProxy' : http_proxy_url,
    'ftpProxy' : None,
    'sslProxy' : https_proxy_url,
    'noProxy' : None,
    'proxyType' : "MANUAL",
    'class' : "org.openqa.selenium.Proxy",
    'autodetect' : False
}

options = {'disable_encoding': True}

driver = webdriver.Chrome('chromedriver', options=chrome_options, desired_capabilities=desired_capabilities, seleniumwire_options=options)

and this is how I'm using wait_for_request:

driver.get(url)
element.click()
request = driver.wait_for_request(response_keyword, timeout=30)
wkeeling commented 2 years ago

Thanks for raising this.

When you use Selenium Wire, you need to pass upstream proxy configuration using the seleniumwire_options argument to the webdriver instead of using desired_capabilities:

options = {
    'proxy': {
        'https': http_proxy_url,
    },
    'disable_encoding': True
}

driver = webdriver.Chrome('chromedriver', options=chrome_options, seleniumwire_options=options)

There's more info in the README if your proxy requires authentication etc.

calum-mcg commented 2 years ago

Thanks @wkeeling for your quick response. Worked a charm!