Feature Request / Question - Mocking with proxies

[janrito]

This fails because the connection is opened to a proxy URL instead of the specified in the entry

from http import HTTPStatus

import pytest
import requests
from mocket import Mocketizer
from mocket.exceptions import StrictMocketException
from mocket.mockhttp import Entry

# setup session
session = requests.Session()
session.proxies = {
    "http": "http://proxy.url/",
    "https": "https://proxy.url/",
}

# mock request to target url
Entry.single_register(
    Entry.GET,
    "http://target.url/",
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

# StrictMocketException: Mocket tried to use the real `socket` module.

We can mock the proxy URL instead

Entry.single_register(
    Entry.GET,
    "http://proxy.url/",  # we mock to proxy url
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

assert res.url == "http://target.url/"

# Success!

However, then it also passes with other URLs

Entry.single_register(
    Entry.GET,
    "http://proxy.url/",  # we mock to proxy url
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://not-the-target.url/")  # different url

assert res.url == "http://target.url/"  # fails

# AssertionError:

I'd prefer to do something like this

Entry.single_register(
    Entry.GET,
    "http://target.url/",
    status=HTTPStatus.OK,
    proxies={"http": "http://proxy.url/", "https": "https://proxy.url/"},
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

with Mocketizer(strict_mode=True):
    with pytest.raises(StrictMocketException):
        res = session.get("http://not-the-target.url/")  # not the right URL

with Mocketizer(strict_mode=True):
    with pytest.raises(StrictMocketException):
        res = requests.get("http://target.url/")  # not through the proxy

[mindflayer]

Hi @janrito, I am sorry I did not get the step below: However, then it also passes with other URLs

I assume you said Mocket did not fail, but then what about the assert? Why asserting against the previous step's URL?

[mindflayer]

This said, I am not sure Mocket should know about proxies, and that's because at socket level you are opening a connection against the proxy, no matter what URL you are telling the proxy to hit for you.

If you tell Mocket to forget about the destination URL, you may end up with broken code just because you have a proxy in the middle.

[mindflayer]

Example: My code uses proxy P to call websites W1 and W2. I want to test that I am calling W1, but I am telling Mocket to forget about that because there is a proxy P in the middle. So, after a refactor, I may have a piece of code that calls W2 instead. My recommendation, since HTTP[S] proxies should work transparently for you - you simply assume the proxy works properly - and that's not relevant when testing, I'd say you should test forgetting about proxies.

[janrito]

Hi @janrito, I am sorry I did not get the step below: However, then it also passes with other URLs

I assume you said Mocket did not fail, but then what about the assert? Why asserting against the previous step's URL?

it's meant to symbolise a typo. In the context of a test, it should fail if I don't request the correct URL.

...

This said, I am not sure Mocket should know about proxies, and that's because at socket level you are opening a connection against the proxy, no matter what URL you are telling the proxy to hit for you.

I get what you mean: mocket just cares about the connection and when using proxies, i'm always opening a connection to the proxy URL and not the final destination.

[mindflayer]

I get what you mean: mocket just cares about the connection and when using proxies, i'm always opening a connection to the proxy URL and not the final destination.

The point is that Mocket knows nothing about the destination URL, it only knows about the actual connection that is opened, which is against the proxy. What Mocket sees is that the client is opening a socket to connect to the proxy.