Post request results in "The server at 127.0.0.1 is taking too long to respond."

[gottogethelp]

I'm trying to call the API of a website using the following code:

from seleniumrequests import Firefox

driver = Firefox()

driver.get("https://www.itftennis.com/en/tournament-calendar/mens-world-tennis-tour-calendar/?categories=All&startdate=2024-01")

driver.request(
    method="POST",
    url="https://www.itftennis.com/tennis/api/TournamentApi/GetDrawsheet",
    json={
        "tournamentId": 1100193157,
        "tourType": "N",
        "weekNumber": 0,
        "matchTypeCode": "S",
        "eventClassificationCode": "M",
    },
)

In the browser window it seems to be trying to load http://127.0.0.1:56330/ which doesn't seem right.

There is a similar issue here however @cryzed puts this down to a firewall issue. That's not the case here as the first get request works just fine.

Any ideas what might be going awry?

Supporting details:

MacOS Sonoma

python: 3.11 selenium-requests: 2.0.3 selenium: 4.17.12 requests: 2.31.0

[cryzed]

It is right. The first .get() is an unmodified Selenium Webdriver-specific method, the second one uses the actual Selenium-Requests .request()-method. (You can easily confirm this by checking out the source-code, it's just a few lines).

When using the latter, it initially does a request to localhost to figure out the "standard headers" sent by the used webdriver (e.g. User-Agent, Accept-Language etc.), so that the low-level requests-based HTTP requests look as similar as possible to what the Selenium webdriver would have sent: that was one of the major motivations behind this library (next to the Cookie synchronization).

I still put it down to a firewall issue or something else system-specific that's impossible for me to debug from where I'm sitting.

[gottogethelp]

Looks like you're right. I've just set up separate Conda environments on two macOS Sonoma machines and installed python 3.11 and selenium-requests. The code above does not work on the machine I was using for the original post but it does on the second machine. Have no idea what it could be but will post back here if I figure it out...

[gottogethelp]

Ok diagnosed the issue - it's my VPN (NordVPN). As soon as I turn it off then the code above works. This is a bit of an issue because I need it on but something for me to take an investigate further. I'll post back if I find a solution unless @cryzed you have any idea on the direction I could take?

[cryzed]

Yes, I have a suspicion:

I think it's possible that NordVPN comes with something like a firewall configuration/"leak protection"/routing that over-eagerly blocks outgoing traffic to the interface we bind to (or tries to route it through the tun-device).

You could try adjusting the code here to server = http.server.HTTPServer(("127.0.0.1", port), HTTPRequestHandler) instead, so that we only bind to localhost specifically -- reflecting on it now, that probably would have been a good idea in the first place.

If that doesn't fix it, I was able to find a few results regarding this issue: here, here and here, which at least seem related.

Can you ping 127.0.0.1 when the VPN is up? Can you ping localhost whent the VPN is up? Are you maybe using the proxy_host parameter with localhost as the value? If so, try changing it to 127.0.0.1.

[gottogethelp]

Thanks a million. I'm out tomorrow but will get to this first thing on Monday morning!