Regression in 3.12.7

[marselester]

Hi Giorgio. Thank you for your work on this library!

It looks like there was a breaking change in https://github.com/mindflayer/python-mocket/compare/3.12.6...3.12.7:

• https://github.com/maxmind/GeoIP2-python/actions/runs/9148591197/job/25152243949#step:5:1083
• https://github.com/maxmind/minfraud-api-python/actions/runs/9225347716/job/25382643724

[mindflayer]

Hi @marselester, thanks for opening this issue. There is something going on with aiohttp I still don't fully understand, but I am aware of it. So far my investigation brought me to see very strong similarities with https://github.com/aio-libs/aiohttp/issues/5582. The latest version of Mocket was a big refactoring, mostly meant to fix https://github.com/mindflayer/python-mocket/pull/234/, and I strongly believe there is nothing wrong with Mocket itself, but of course I am open to help with this.

[mindflayer]

I see from your CI that the same code works for the most recent versions of Python, which coincides with my findings.

[marselester]

Thank you for looking into this issue ❤️

[mindflayer]

Hey @marselester, if you manage to write a few lines able to replicate the issue I'll be more than happy to help with that, even if I suspect it's not related to Mocket itself.

[marselester]

Maybe this would help. I know it doesn't narrow down the problem greatly, but it could be a starting point.

$ pyenv local 3.10.13
$ virtualenv venv
$ . venv/bin/activate
(venv) $ pip install -e git+https://github.com/maxmind/GeoIP2-python.git@559f145915c811c2ea93201195ad2139debce760#egg=geoip2
(venv) $ pip install mocket==3.12.7
(venv) $ pip freeze
aiohttp==3.9.5
aiosignal==1.3.1
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.2.2
charset-normalizer==3.3.2
decorator==5.1.1
frozenlist==1.4.1
-e git+https://github.com/maxmind/GeoIP2-python.git@559f145915c811c2ea93201195ad2139debce760#egg=geoip2
httptools==0.6.1
idna==3.7
maxminddb==2.6.1
mocket==3.12.7
multidict==6.0.5
python-magic==0.4.27
requests==2.32.3
urllib3==2.2.1
yarl==1.9.4
(venv) $ python mytest.py

Trace

``` /blah/.pyenv/versions/3.10.13/lib/python3.10/asyncio/selector_events.py:746: ResourceWarning: unclosed self._protocol = None ResourceWarning: Enable tracemalloc to get the object allocation traceback E ====================================================================== ERROR: test_200_error (__main__.TestAsyncClient) ---------------------------------------------------------------------- Traceback (most recent call last): File "/blah/blah/venv/lib/python3.10/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) File "/blah/blah/venv/lib/python3.10/site-packages/mocket/mocket.py", line 736, in wrapper return test(*args, **kwargs) File "/blah/blah/mytest.py", line 33, in test_200_error self.run_client(self.client.country("1.1.1.1")) File "/blah/blah/mytest.py", line 19, in run_client return self._loop.run_until_complete(v) File "/blah/.pyenv/versions/3.10.13/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/blah/blah/venv/src/geoip2/geoip2/webservice.py", line 307, in country await self._response_for("country", geoip2.models.Country, ip_address), File "/blah/blah/venv/src/geoip2/geoip2/webservice.py", line 346, in _response_for async with await session.get(uri, proxy=self._proxy) as response: File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/client.py", line 608, in _request await resp.start(conn) File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 971, in start with self._timer: File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/helpers.py", line 735, in __exit__ raise asyncio.TimeoutError from None asyncio.exceptions.TimeoutError ---------------------------------------------------------------------- Ran 1 test in 60.079s FAILED (errors=1) ```

Test

```python import asyncio import unittest from mocket.plugins.httpretty import httpretty, httprettified from geoip2.errors import GeoIP2Error from geoip2.webservice import AsyncClient class TestAsyncClient(unittest.TestCase): def setUp(self): self._loop = asyncio.new_event_loop() self.client = AsyncClient(42, "abcdef123456") def tearDown(self): self._loop.run_until_complete(self.client.close()) self._loop.close() def run_client(self, v): return self._loop.run_until_complete(v) @httprettified def test_200_error(self): httpretty.register_uri( httpretty.GET, "https://geoip.maxmind.com/geoip/v2.1/country/1.1.1.1", body="", status=200, content_type="application/vnd.maxmind.com-country+json; charset=UTF-8; version=1.0", ) with self.assertRaisesRegex( GeoIP2Error, "could not decode the response as JSON" ): self.run_client(self.client.country("1.1.1.1")) if __name__ == "__main__": unittest.main() ```

[marselester]

I took an example from your README and it also failed with asyncio.exceptions.TimeoutError.

(venv) $ python t.py

Test

```python import json import aiohttp import asyncio import unittest from mocket.plugins.httpretty import httpretty, httprettified class AioHttpEntryTestCase(unittest.TestCase): @httprettified def test_https_session(self): url = 'https://httpbin.org/ip' httpretty.register_uri( httpretty.GET, url, body=json.dumps(dict(origin='127.0.0.1')), ) async def main(l): async with aiohttp.ClientSession( loop=l, timeout=aiohttp.ClientTimeout(total=3) ) as session: async with session.get(url) as get_response: assert get_response.status == 200 assert await get_response.text() == '{"origin": "127.0.0.1"}' loop = asyncio.new_event_loop() loop.set_debug(True) loop.run_until_complete(main(loop)) if __name__ == '__main__': unittest.main() ```

Trace

``` /blah/.pyenv/versions/3.10.13/lib/python3.10/asyncio/selector_events.py:746: ResourceWarning: unclosed self._protocol = None ResourceWarning: Enable tracemalloc to get the object allocation traceback E ====================================================================== ERROR: test_https_session (__main__.AioHttpEntryTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/blah/blah/venv/lib/python3.10/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) File "/blah/blah/venv/lib/python3.10/site-packages/mocket/mocket.py", line 736, in wrapper return test(*args, **kwargs) File "/blah/blah/t.py", line 30, in test_https_session loop.run_until_complete(main(loop)) File "/blah/.pyenv/versions/3.10.13/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/blah/blah/t.py", line 24, in main async with session.get(url) as get_response: File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/client.py", line 1197, in __aenter__ self._resp = await self._coro File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/client.py", line 608, in _request await resp.start(conn) File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 971, in start with self._timer: File "/blah/blah/venv/lib/python3.10/site-packages/aiohttp/helpers.py", line 735, in __exit__ raise asyncio.TimeoutError from None asyncio.exceptions.TimeoutError ---------------------------------------------------------------------- Ran 1 test in 3.011s FAILED (errors=1) ```

[mindflayer]

The strange thing is that it works well with Python 3.11+.

[mindflayer]

OK, there was indeed something strange when using aiohttp with HTTPS urls. Weird that I did not see it happening with Python 3.11+, but still a problem to fix.

[mindflayer]

Here is a version which fixes it: https://pypi.org/project/mocket/3.12.8/

[marselester]

Awesome, thanks a lot!