Cannot inject HTTPX client as pytest fixture

[gregbrowndev]

Hi,

Thanks for fixing the previous issue super quickly. However, I've noticed a json.decoder.JSONDecodeError problem when you inject HTTPX' AsyncClient as a pytest fixture. If you put the @mocketize decorator on the fixture it fixes the problem. However, this seems a bit odd.

import asyncio
import json

import httpx
import pytest
from httpx import AsyncClient
from mocket import mocketize
from mocket.mockhttp import Entry

@pytest.fixture
def httpx_client() -> AsyncClient:
    # Note: should use 'async with'
    return httpx.AsyncClient()

async def send_request(client: AsyncClient, url: str) -> dict:
    r = await client.get(url)
    return r.json()

@mocketize
def test_mocket(
    httpx_client: AsyncClient
):
    # works if you define httpx_client locally:
    # httpx_client = httpx.AsyncClient()
    url = "https://example.org/"
    data = {"message": "Hello"}

    Entry.single_register(
        Entry.GET,
        url,
        body=json.dumps(data),
        headers={'content-type': 'application/json'}
    )

    loop = asyncio.get_event_loop()

    coroutine = send_request(httpx_client, url)
    actual = loop.run_until_complete(coroutine)

    assert data == actual

Stacktrack

``` /Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py:646: in run_until_complete return future.result() test_mocket.py:22: in send_request return r.json() /Users/gregorybrown/Library/Caches/pypoetry/virtualenvs/tariff-management-6yv2RoDp-py3.10/lib/python3.10/site-packages/httpx/_models.py:743: in json return jsonlib.loads(self.text, **kwargs) /Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/json/__init__.py:346: in loads return _default_decoder.decode(s) /Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/json/decoder.py:337: in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = s = '\n\n\n \n\n \n \n

More information...

\n

[mindflayer]

From your stacktrace I don't understand how mocket would be involved with this error. It seems like something happening on httpx side.

Could you please comment all mocket's code?

[mindflayer]

Also, I see you are not using async_mocketize but your code looks like async to me.

[gregbrowndev]

Thanks for the quick reply.

Maybe as a bit of context. This test is trying to emulate a sync function that uses httpx to make a bunch of concurrent requests. Basically calling async from a sync function. It does this by putting the coroutines on the event loop like in the test. The sync function is buried deep in the app, so this is to contain the async and prevent us converting the whole app to async in one go.

The pytest is synchronous, hence it doesn't require async_mocketize. For example, this test passes simply because httpx_client is instantiated locally:

@mocketize
def test_mocket():
    httpx_client = httpx.AsyncClient()
    url = "https://example.org/"
    data = {"message": "Hello"}

    Entry.single_register(
        Entry.GET,
        url,
        body=json.dumps(data),
        headers={'content-type': 'application/json'}
    )

    loop = asyncio.get_event_loop()

    coroutine = send_request(httpx_client, url)
    actual = loop.run_until_complete(coroutine)

    assert data == actual

The registered mocks using Entry.single_register are being returned by the httpx requests as expected with this set up. The only problem is the test breaks if you use a fixture.

[gregbrowndev]

From your stacktrace I don't understand how mocket would be involved with this error.

I'll look into this, maybe this is the real problem!

Thanks

[gregbrowndev]

It doesn't look specifically like a httpx issue. This test passes (calling out to a real http API):

import asyncio

import httpx
import pytest
from httpx import AsyncClient

async def send_request(client: AsyncClient, url: str) -> dict:
    r = await client.get(url)
    return r.json()

@pytest.fixture
def httpx_client() -> AsyncClient:
    return httpx.AsyncClient()

def test_async(httpx_client: AsyncClient):
    url = "https://dummyjson.com/products/1"

    loop = asyncio.get_event_loop()
    coroutine = send_request(httpx_client, url)
    actual = loop.run_until_complete(coroutine)

    assert "id" in actual

However, if you add the decorator, you get the error below (different to the original one)

Stacktrace

``` Traceback (most recent call last): File "/Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/concurrent/futures/_base.py", line 330, in _invoke_callbacks callback(self) File "/Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/asyncio/futures.py", line 398, in _call_set_state dest_loop.call_soon_threadsafe(_set_state, destination, source) File "/Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 801, in call_soon_threadsafe self._write_to_self() File "/Users/gregorybrown/.pyenv/versions/3.10.4/lib/python3.10/asyncio/selector_events.py", line 135, in _write_to_self csock.send(b'\0') File "/Users/gregorybrown/Library/Caches/pypoetry/virtualenvs/tariff-management-6yv2RoDp-py3.10/lib/python3.10/site-packages/mocket/mocket.py", line 391, in send entry = self.get_entry(data) File "/Users/gregorybrown/Library/Caches/pypoetry/virtualenvs/tariff-management-6yv2RoDp-py3.10/lib/python3.10/site-packages/mocket/mocket.py", line 258, in get_entry return Mocket.get_entry(self._host, self._port, data) File "/Users/gregorybrown/Library/Caches/pypoetry/virtualenvs/tariff-management-6yv2RoDp-py3.10/lib/python3.10/site-packages/mocket/mocket.py", line 430, in get_entry host = host or Mocket._address[0] AttributeError: type object 'Mocket' has no attribute '_address' ```

[mindflayer]

This looks like a mocket one! :) I'll have a look at it ASAP.

[mindflayer]

First of all, the error related to JSON decoding is a symptom of something wrong. It looks like mocket is not intercepting - or serving - the call. You can see it from the following screenshot: It's exactly the content of the homepage of the website it was supposed to mock (example.org). When you write snippets like that I suggest you to use fake URLs. Now I'm trying to understand why the client from the fixture does not work properly.

[mindflayer]

It looks like the client from the fixture is living in a non-perfectly-mocked reality. Mocket is doing right but, for some reason, after it serves the response, a real call happens.

Like you said, the fix is in enabling mocket from inside the fixture like I did, or using the decorator like you mentioned before.

import json

import httpx
import pytest

from mocket.mockhttp import Entry
from mocket import Mocketizer

@pytest.fixture
def httpx_client() -> httpx.AsyncClient:
    with Mocketizer():
        yield httpx.AsyncClient()

@pytest.mark.asyncio
async def test_httpx(httpx_client):
    url = "https://foo.bar/"
    data = {"message": "Hello"}

    Entry.single_register(
        Entry.GET,
        url,
        body=json.dumps(data),
        headers={"content-type": "application/json"},
    )

    async with httpx_client as client:
        response = await client.get(url)

        assert response.json() == data

[mindflayer]

I've just added the above test to mocket's test suite. You can have a look at the linked PR.