Colin-b / pytest_httpx

pytest fixture to mock HTTPX
https://colin-b.github.io/pytest_httpx/
MIT License
359 stars 30 forks source link

Allow usage with `httpx.MockTransport` #167

Open CarliJoy opened 1 day ago

CarliJoy commented 1 day ago

In certain use cases, pytest_httpx is used only to mock a single httpx client, specifically one responsible for connecting to a cluster of servers for service A. However, while mocking this specific client, we still want the flexibility to make normal, non-mocked requests to other services without impacting the entire environment.

Previously, we initialized HTTPXMock directly, but with recent changes, this is no longer supported. While the focus on clean interfaces is understandable, this change limits the ability to easily create single mocked clients while keeping other requests unaltered.

To address this, I propose adding support for a MockTransport, as outlined below:

TCallable = TypeVar("TCallable", bound=Callable)

def copy_docs(source_func: Callable) -> Callable[[TCallable], TCallable]:
    def apply_docs(target_func: TCallable) -> TCallable:
        target_func.__doc__ == source_func.__doc__

class MockedTransport(httpx.MockTransport):
    def __init__(
        self,
        assert_all_responses_were_requested: bool = True,
        assert_all_requests_were_expected: bool = True,
        can_send_already_matched_responses: bool = False,
    ):
        _pool = None  # enusre _proxy_url does not fail
        options = _HTTPXMockOptions(
            can_send_already_matched_responses=can_send_already_matched_responses,
            assert_all_responses_were_requested=assert_all_responses_were_requested,
            assert_all_requests_were_expected=assert_all_requests_were_expected,
        )
        self.mock = HTTPXMock(options)
        super().__init__(lambda request: self.mock._handle_request(self, request))

    # TODO copy call signature
    # see https://stackoverflow.com/a/71968448/3813064 or
    #     https://github.com/python/cpython/pull/121693
    @copy_docs(HTTPXMock.add_response)
    def add_response(self, *args, **kwargs) -> None:
        self.mock.add_response(*args, **kwargs)

    @copy_docs(HTTPXMock.add_callback)
    def add_callback(self, *args, **kwargs) -> None:
        self.mock.add_callback(*args, **kwargs)

    @copy_docs(HTTPXMock.add_exception)
    def add_exception(self, *args, **kwargs) -> None:
        self.mock.add_exception(*args, **kwargs)

The MockedTransport class extends integrate smoothly with HTTPXMock, allowing targeted client mocking while enabling other clients to make live requests as usual.

Colin-b commented 1 day ago

Hello @CarliJoy ,

I am not sure to follow, why can't this be achieved with the should_mock option set on the full test suite?

CarliJoy commented 18 hours ago

Hi colin-b,

Got it, we were using an older version with only the non_mocked_hosts option. Thanks for the heads-up!

We still need to patch a specific client for our use case. We provide a pytest fixture as a plugin for our service, and users often want to mock additional services with pytest_httpx.

However, using a global mocking state creates issues:

Additionally, different service tests might require varying options for assert_all_responses_were_requested, assert_all_requests_were_expected, or can_send_already_matched_responses. Without the ability to mock specific httpx clients, handling these cases is challenging.

Colin-b commented 9 hours ago

Your should_mock callable could rely on some shared variable (a list of urls, hosts, headers, etc... that you would base yourself upon to know if you should mock or not) that your clients would be free to update (thus keeping the previous setup you want to use as common). The value is evaluated at request time so this should work just fine as in the following:

shared_mocked_hosts = ["default_mocked_host"]

def should_mock(request: httpx.Request) -> bool:
    return request.url.host in shared_mocked_hosts

The other options are evaluated only for mocked requests so it should be fine as well I assume?