Koed00 / django-q

A multiprocessing distributed task queue for Django
https://django-q.readthedocs.org
MIT License
1.83k stars 285 forks source link

Supposedly "optional" redis requirement is a default dependency? #696

Open coredumperror opened 2 years ago

coredumperror commented 2 years ago

I'm not sure what's causing this (where's the setup.py in this repo...?), but installing django-q into my virtualenv just now, in my first attempts to check it out and see if it'll work for my project, caused it to install redis v3.5.3. This was undesirable, because I was intending to use the ORM as my broker, and already had a newer redis package installed for another purpose. So this installation removed that newer redis and shoved an older version into my virtualenv, which was quite undesirable.

It seems to me that if redis is intended to be an optional dependency, it should not be installed by default.

coredumperror commented 2 years ago

Actually, this is worse than I thought. Once I put django-q[sentry]==1.3.9 into my requirements file, I was suddenly unable to build my project's Docker image at all, as pip throws this error:

#9 56.12
#9 56.12 The conflict is caused by:
#9 56.12     The user requested redis==4.3.4
#9 56.12     django-redis 5.2.0 depends on redis!=4.0.0, !=4.0.1 and >=3
#9 56.12     django-q 1.3.9 depends on redis<4.0.0 and >=3.5.3
#9 56.12
#9 56.12 To fix this you could try to:
#9 56.12 1. loosen the range of package versions you've specified
#9 56.12 2. remove package versions to allow pip attempt to solve the dependency conflict
#9 56.12
#9 56.12 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Considering that I don't even want to use redis as my Django Q broker, I consider this a rather serious bug in Django Q, and will actually have to abandon it entirely until/unless this gets fixed.

Further experimentation leads me to believe that the reason this is happening is the changeover to "poetry" in Django Q 1.3.7. I can build my Docker image just fine when using 1.3.6, thankfully.

GDay commented 1 year ago

You are right. For some reason poetry adds this to the required ones, even though it's marked as optional. I have now added it to the testing group and that seems to fix the issue:

Step 6/6 : RUN pip install django django-q2==1.4.2
 ---> Running in ebe38fb8e0b6
Collecting django
  Downloading Django-4.1.2-py3-none-any.whl (8.1 MB)
Collecting django-q2==1.4.2
  Downloading django_q2-1.4.2-py3-none-any.whl (93 kB)
Collecting django-picklefield<4.0,>=3.1
  Downloading django_picklefield-3.1-py3-none-any.whl (9.5 kB)
Collecting asgiref<4,>=3.5.2
  Downloading asgiref-3.5.2-py3-none-any.whl (22 kB)
Collecting sqlparse>=0.2.2
  Downloading sqlparse-0.4.3-py3-none-any.whl (42 kB)
Installing collected packages: sqlparse, asgiref, django, django-picklefield, django-q2
Successfully installed asgiref-3.5.2 django-4.1.2 django-picklefield-3.1 django-q2-1.4.2 sqlparse-0.4.3

You will need to download my fork though, which you can find here: https://github.com/GDay/django-q2 It's compatible with django-q, you don't need to make any changes to your code. One small caveat: if you use any of the commands, like qmoniter, then you also need to install blessed (I have made it optional, so it will be removed when you remove django-q and not be installed when you install django-q2).