driver.requests empty when using with safari

[circuitsacul]

I am trying to get all requests made when I go to a website. When I use Chrome as the webdriver, it works fine. However, if I try to use safari, driver.requests is empty.

I tried the example code with Chrome and Safari. It worked with the first but not with the second. I tried adding a 10-second delay to see if maybe it needed more time, but it still didn't work.

Safari works fine with selenium wire, it will go to websites and click buttons just fine, but driver.requests is always empty.

Edit: I am including example code to show the error:

from seleniumwire import webdriver
import sys

safaridriver = webdriver.Safari(seleniumwire_options={'port': 12345})
chromedriver = webdriver.Chrome()

try:
    safaridriver.get('https://google.com')
    print(f"Length of safaridriver.requests: {len(safaridriver.requests)}")
    chromedriver.get('https://google.com')
    print(f"Length of chromedriver.requests: {len(chromedriver.requests)}")
    print(f"safaridriver.requests == chromedriver.requests: {safaridriver.requests == chromedriver.requests}")
finally:
    safaridriver.quit()
    chromedriver.quit()

Output:

Length of safaridriver.requests: 0
Length of chromedriver.requests: 24
safaridriver.requests == chromedriver.requests: False

I do not get any errors, (I tried without the try statement). It also goes to google.com on both browsers, and I can even use safari to click buttons. But driver.requests is still empty. I also tried with a very long delay, same problem. Please help.

Edit 2 I just read the guidelines lol. I am using a MacBook air running on MacOS Catalina, I am using python 3.7 and the latest version of selenium wire (I downloaded a few days ago)

[wkeeling]

You've run through these steps? https://github.com/wkeeling/selenium-wire/blob/master/safari_setup.rst

[circuitsacul]

I ran through the steps. The last step is what won't work for me. When I set the proxies, I get a "unable to connect" error when using safari with selenium or just as I normally use it. I marked the check next to both the 'http' and 'https' proxies, set the server to 'localhost', and the port to 12345. This may or may not matter but I also set the port inside the script to '12345'.

[circuitsacul]

Do you know what I am doing wrong? @wkeeling

[wkeeling]

Sorry for the delay in replying.

As you've discovered, using Safari with Selenium Wire is less than optimal. Selenium Wire works by automatically configuring the browser's proxy settings so that they point to the local proxy server running inside Selenium Wire. That means that as the browser makes requests, they flow through Selenium Wire and can be captured.

Except... that automatic proxy configuration is only supported by Chrome and Firefox and not Safari. With Safari we have to manually configure the proxy. This means that once Safari has been configured to use Selenium Wire's proxy on a port (e.g. 12345), then the browser will try and send all requests to that port when you're using it. If Selenium Wire isn't running, and thus not listening on port 12345, you'll see "unable to connect" messages. For regular browsing (and regular Selenium) you'll need to go back and disable the proxy.

However, once you've configured the proxy manually, Safari then ought to work with Selenium Wire. What happens when you run Selenium Wire after you've configured the proxy?

[mayank1004]

Hi @wkeeling ,

Appreciate you looking into this issue.

Please let me know if any update on the above. I am facing the same issue: when Safari is used, driver.requests is returning empty list (when using Chrome, driver.requests returning some values for the same script). I had successfully run through the steps mentioned in https://github.com/wkeeling/selenium-wire/blob/master/safari_setup.rst I am using python 3.8.0, Safari Version 13.1, macOS Catalina version 10.15.4. Selenium-wire I am using 2.1.1.

Thanks

[circuitsacul]

@mayank1004 My problem was that I didn't do the steps correctly... It's a real pain to do. I don't know if this will help you, but this is what went wrong for me:

After completing the last step, I was getting the error "unable to connect". This is because once the proxy is configured, it tries to connect to selenium wire. And if it's not running, it can't connect.

[twixdouble]

https://github.com/wkeeling/selenium-wire/blob/master/safari_setup.rst shows 404 - could you please share an actual link

[wkeeling]

@twixdouble Safari isn't technically supported at the moment, although it should still work with the latest version of Selenium Wire. You can find the original link here: https://github.com/wkeeling/selenium-wire/blob/2.1.2/safari_setup.rst

The instructions are still valid, except there is one step missing. When you configure Selenium Wire, you need to specify the same port number that you used when you configured the proxy in Safari - e.g.

driver = webdriver.Safari(seleniumwire_options={'port': 12345})  # Safari must also be configured with this port

Also remember that once you've configured the proxy in Safari, it will try and use that proxy for all internet access. So if you use Safari for regular internet browsing, you'll need to unconfigure the proxy for that.