Closed WisdomPill closed 2 years ago
Hi @WisdomPill, I am having a look at that. Thanks for the feedback.
Before starting, I must say that usually when testing Django caching you use DummyCache
(https://docs.djangoproject.com/en/3.1/topics/cache/#dummy-caching-for-development), but that of course has nothing to do with mocket
failing, which has to be fixed. Maybe you could take it as a workaround for now.
The other thing is: I have to create a dummy project with Django, your script cannot work without it.
Hello @mindflayer,
well, I can't use DummyCache
because I am using some features of django-redis
that are not standard in django cache, like a redis' Lock
.
For now my look around is to run a dummy test that is a mere copy of test_b_random_url
of the example, so that alphabetically runs before.
I can create a dummy project with the test case in place if needed.
What version of mocket
are you using?
Nothing is failing on my machine.
I launched a env DJANGO_SETTINGS_MODULE=app.app.settings pytest test.py
Django's app has been created with a simple django-admin startproject app
. Probably it is not failing because I did not configure Django caching.
What redis
version are you using? I had problems in the past when the author changed the way it manages connections. I need to isolate the problem, since it's clear that it has nothing to do with Django.
All mocket
tests are passing with the latest version of redis
:
[...]
----------- coverage: platform linux, python 3.8.6-final-0 -----------
Name Stmts Miss Cover Missing
--------------------------------------------------------------------
mocket/__init__.py 3 0 100%
mocket/async_mocket.py 12 2 83% 7, 10
mocket/mocket.py 392 7 98% 71-73, 87, 102, 189, 309
mocket/mockhttp.py 124 0 100%
mocket/mockredis.py 52 0 100%
mocket/plugins/__init__.py 0 0 100%
mocket/plugins/httpretty/__init__.py 71 0 100%
mocket/plugins/httpretty/core.py 2 0 100%
mocket/plugins/pook_mock_engine.py 43 0 100%
mocket/utils.py 20 0 100%
--------------------------------------------------------------------
TOTAL 719 9 99%
=========================================================================== 140 passed, 12 warnings in 9.71s ===========================================================================
(.venv) drizzt@mindflayer ~/r/python-mocket (master)> pip freeze | grep redis
redis==3.5.3
I am using the same redis version. I am trying to setup a dummy project. Here is my pipfile with the dependencies needed if you want to try in the meantime.
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[dev-packages]
[packages]
django = "3.1.2"
django-redis = "==4.12.1"
mocket = "==3.9.40"
redis = "==3.5.3"
requests = "==2.24.0"
I am trying to setup a dummy project.
Thanks, that would help. :)
With just the requirements of my previous comment it seems to work, but if I copy the requirements of the project in which there is the issue it happens again. I stripped down the dependencies to see if the issue is still there and it is. I left most of the dependencies that have to do with redis. I will try to remove more dependencies, in the meantime here is the link to the repo, you can have a look at the actions, I can add you to the repo if you want to use the action that is implemented there. I can also add a matrix to see if with other versions of packages, python or redis it might differ, but at the moment I don't think so.
It has to do with hiredis
, I will read further probably tomorrow.
Thanks for the quick response.
I will let you know why it happens and if there is a quick fix to make mocket
work in harmony with hiredis
It has to do with
hiredis
, I will read further probably tomorrow.
That's quite strange, AFAIK hiredis
has nothing to do with networking and it's for speeding up the Redis protocol parsing.
Let me know in case you feel I can be of help.
I did some digging and removed django.
I found out that redis-py
gives the error only when hiredis
is available because it tries to use the socket to read (have a look here) the response. In fact now it gives the error back all the time regardless of the orther of the tests.
I never used mocket for redis so I do not know how it mocks behind the scenes, can you help me understand why even if using @mocketized
I can access redis? I thought pocket will mock all the sockets, in fact that is the reason why the test fails.
I updated the project, maybe you can have a look at it?
can you help me understand why even if using @mocketized I can access redis?
When using mocket
, you are literally mocking the socket
module with the first acting as a proxy. So, all the calls which are mapped by specific Entry
are mocked, all the rest are supposed to "fall-back" to their real sockets. In this case mocket
is failing on using the real socket when hiredis
is in the middle.
At the moment the alternatives you have are:
mocket
as a context manager and mock only your HTTP call as follows;
from mocket import Mocket, Mocketizer
def test_random_url_with_redis(redis_client: Redis): url = "http://www.example.com" Entry.single_register(Entry.GET, url)
redis_client.set("key", "value")
with Mocketizer():
requests.get(url)
assert Mocket.has_requests()
- removing `hiredis` when testing;
- mocking all your Redis calls with `patch` from `unittest.mock` or any other solution that prevents Python to opening a socket.
I'll do my best to understand how to fix the error, but sometimes it takes a while.
Okay, thanks for the explanation.
I already found some workaround in my projects, thanks for pointing out some solutions though.
I just like mocket
and want to help by reporting this issue.
P.S. I wanted to make another point in why I am not using DummyCache
, I prefer having the exact environment of staging and production during testing, so DummyCache
is something that I never even thought of using to not incur in any preventable issue.
I understand your point of view and I believe it makes sense, especially if your logic can be different depending on something cached or not, but also consider that a cache, by definition, should not be a "blocker" and your application should be able to work when Redis is not available, it's just there as an additional layer which helps to speed up the flow. This said, the first alternative I presented to you (context manager) implies you have a working Redis in your testing environment. If so, everything will work properly.
I just like mocket and want to help by reporting this issue.
Thank you so much for your help, btw.
Describe the bug I have a
django
based software that makes http requests usingrequests
and usesdjango-redis
to store some data of what it has fetched. I have noticed that if the first network call in aTestCase
is forredis
thenmocket
mocks redis, otherwise it mocks onlyrequests
.To Reproduce
will throw an exception
If you swap names of the test functions it works as expected.
Expected behavior Choose what to mock with some flag.
Additional context It happens in macOS and ubuntu, with python3.7 and also 3.8