wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

Catch "Invalid proxy server credentials supplied" Exception #548

Open yanivps opened 2 years ago

yanivps commented 2 years ago

Hi, I am getting a list of proxy servers from some API and apply them dynamically to the driver.proxy dictionary The problem is that sometimes, I can get a proxy with invalid credentials which leads to the following message in the log file: Invalid proxy server credentials supplied which comes from the file: thirdparty/mitmproxy/server/protocol/http.py and I am trying to find way to catch these errors whenever I call driver.get(...) either with or without request_interceptor but I couldn't find a way. I've noticed that the request_interceptor does not get called when the proxy credentials are invalid. Is there a way to handle this?

Any help would be much appriciated!

vladtermene commented 2 years ago

Hello. I am having the same problem. I wish it would throw an exception when proxy credentials are invalid rather than just print the message.

What I am currently doing is a while loop, and I exit it only when driver.get() successfully executes. Furthermore, the website is giving me the IP, so I make sure the IP is the same as the one supplied in proxy. But this is rather slow.

wkeeling commented 2 years ago

This is tricky since the error with the invalid proxy server credentials happens inside the connection handling threads which are separate to the thread that you run driver.get(). Therefore changing Selenium Wire to raise an exception rather than printing a message won't work because your code will never be able to catch the exception. The interceptor isn't called because the request hasn't yet been made - the connection is still being negotiated (and failing).

One idea for dealing with this scenario could be to pass a callback function which gets notified in the event of invalid proxy credentials. Support for this would need implementing in Selenium Wire, but it could work something like this:

# Design idea - not currently implemented in Selenium Wire

def on_connection_error(error):  # Gets called when the proxy credentials are invalid
    do_something_with(error)

driver = webdriver.Chrome(seleniumwire_options={
    'proxy': {
        'https': 'https://user:pass@hostname',
        'on_connection_error': on_connection_error  # Pass the callback function here
    },
})