Loosen the version constraint of redis to avoid dependency conflicts

[NeolithEra]

Hi, redis-py-cluster locked the version constraint of redis as redis==2.10.6, which leads to a troubling scenario that its direct downstream project [arxiv-auth, scrapy-redis-bloomfilter-block-cluster,scrapy-redis-cluster,django-redis-cluster,dedis-cluster,redis-pubsub-dict,async-bowl,flask-cache-redis-cluster,iprange-python] has to lock redis.

Could you please loosen the version constraint of redis? Benefit of this is that users using both of [arxiv-auth, scrapy-redis-bloomfilter-block-cluster,scrapy-redis-cluster,django-redis-cluster,dedis-cluster,redis-pubsub-dict,async-bowl,flask-cache-redis-cluster,iprange-python] and redis can upgrade their third party libraries in a timely manner to reduce technical debts.

Solution

The dependency trees of your project and affected downstream projects are shown as follows. Taking the version constraints of upstream and downstream projects into comprehensive consideration, you can

1. Loosen redis==2.10.6 to be redis>=2.10.6.
2. Loosen redis==2.10.6 to be redis==*.
3. Try to add an upper bound for redis version constraint, according to your compatibility.

@Grokzen Please let me know your choice. I can submit a PR to fix this issue.

Thanks for your attention. Best, Neolith

[NeolithEra]

Dependency tree-----

redis-py-cluster-1.3.6
| +-redis(version range:==2.10.6)

Direct downstream project:

arxiv-auth-0.3.5rc2
| +-arxiv-base(version range:*)
| +-flask-sqlalchemy(version range:*)
| | +-flask(version range:>=0.10)
| | | +-click(version range:>=5.1)
| | | +-itsdangerous(version range:>=0.24)
| | | +-jinja2(version range:>=2.10.1)
| | | +-werkzeug(version range:>=0.15)
| | +-sqlalchemy(version range:>=0.8.0)
| +-mimesis(version range:*)
| +-mysqlclient(version range:*)
| +-pycountry(version range:*)
| +-pyjwt(version range:*)
| +-python-dateutil(version range:*)
| | +-six(version range:>=1.5)
| +-redis(version range:==2.10.6)
| +-redis-py-cluster(version range:==1.3.6)
| | +-redis(version range:==2.10.6)
| +-sqlalchemy(version range:*)

scrapy-redis-bloomfilter-block-cluster-1.0.1
| +-redis(version range:>=2.10)
| +-redis-py-cluster(version range:>=1.3.4)
| | +-redis(version range:==2.10.6)
| +-scrapy(version range:>=1.4)
| | +-queuelib(version range:*)
| | +-twisted(version range:>=13.1.0,<=19.2.0python)
| | +-twisted(version range:>=13.1.0python)
| | +-w3lib(version range:>=1.17.0)
| +-six(version range:>=1.5.2)
| +-twisted(version range:==18.9.0)

scrapy-redis-cluster-0.4
| +-redis(version range:>=2.10)
| +-redis-py-cluster(version range:>=1.3.4)
| | +-redis(version range:==2.10.6)
| +-scrapy(version range:>=1.0)
| | +-queuelib(version range:*)
| | +-twisted(version range:>=13.1.0,<=19.2.0python)
| | +-twisted(version range:>=13.1.0python)
| | +-w3lib(version range:>=1.17.0)
| +-six(version range:>=1.5.2)

scrapy-redis-cluster

django-redis-cluster-1.0.0
| +-django(version range:>=1.9.6)
| +-msgpack-python(version range:>=0.4.7)
| +-redis(version range:>=2.10.5)
| +-redis-py-cluster(version range:>=1.2.0)
| | +-redis(version range:==2.10.6)

`` dedis-cluster-1.0.2 | +-django(version range:>=1.9.6) | +-msgpack-python(version range:>=0.4.7) | +-redis(version range:>=2.10.5) | +-redis-py-cluster(version range:>=1.2.0) | | +-redis(version range:==2.10.6)

redis-pubsub-dict-0.0.2 | +-msgpack(version range:) | +-pylru(version range:) | +-redis(version range:) | +-redis-py-cluster(version range:) | | +-redis(version range:==2.10.6)

async-bowl-0.0.1 | +-mugen(version range:>=0.4.3) | +-pika(version range:>=1.0.1) | +-redis(version range:>=2.10.5) | +-redis-py-cluster(version range:>=1.3.6) | | +-redis(version range:==2.10.6)

iprange-python-0.0.7 | +-netaddr(version range:>=0.7.19) | +-redis(version range:>=2.10.6) | +-redis-py-cluster(version range:>=1.3.4) | | +-redis(version range:==2.10.6)

flask-cache-redis-cluster-0.0.5 | +-flask(version range:) | | +-click(version range:>=5.1) | | +-itsdangerous(version range:>=0.24) | | +-jinja2(version range:>=2.10.1) | | | +-markupsafe(version range:>=0.23) | | +-werkzeug(version range:>=0.15) | +-redis(version range:) | +-redis-py-cluster(version range:) | | +-redis(version range:==2.10.6) | +-werkzeug(version range:)

iprange-python-0.0.7 | +-netaddr(version range:>=0.7.19) | +-redis(version range:>=2.10.6) | +-redis-py-cluster(version range:>=1.3.4) | | +-redis(version range:==2.10.6)

[NeolithEra]

See the related issue ： https://github.com/leffss/ScrapyRedisBloomFilterBlockCluster/issues/1

[Grokzen]

Looking at your three solutions

- Loosen redis==2.10.6 to be redis>=2.10.6.
- Loosen redis==2.10.6 to be redis==*.
- Try to add an upper bound for redis version constraint, according to your compatibility.

Nr 1 will not happen in the 1.3.x line, it is about compatibility issues and that upstream versions can't be guaranteed to be supported in that line.

Loosen it to * is not a good solution as that would just open up for compatibility issues as well as any new upstream release would be installed and not really supported or compatible or tested.

Nr 3 is what is currently the setup for the 1.3.x line. Yes it could technically be relaxed down to 2.10.3 to 2.10.6 versions. But since 2.10.6 is the latest version in the 2.10.x line it will remain like that.

The 3.x.x version line changed around some things that makes it not possible to just drop it in to 1.3.x and hade it work out.

[NeolithEra]

@Grokzen So what your suggested solution for this issue?

[Grokzen]

tag 2.0.0 helps to improve the situation. When i can dig deeper into the code for redis-py releases 3.1.x to 3.3.x i can see how many versions that can be supported and what/when/where it breaks with older releases. I have not read up on the release notes for redis-py what triggered the jumps to 3.1, 3.2, 3.3 major version lines, if there is anything breaking in them or not.

My plan is to at least have support for 2 or maybe 3 major versions of redis-py in any given major version of this repo, but i can't guarantee that, it is just a goal i will aim for.

[bobbyrward]

A better option may be to vendor redis-py in redis-py-cluster. If the project is always going to have a tight version constraint, that's likely the only way to allow this libraryt to live in the same project as other libraries that use redis. It's a bit of a hassle for sure, but it's likely the only way to really deal with this.

[Grokzen]

Vendoring the upstream project would only solve the issue where you want to run a redis-py in a non-cluster and a cluster mode inside the same project/code.

Right now, vendorizing the upstream project is not really an option i want to go with. And i can't unlock the versioning to broad either since i can't guarantee support for new major version lines of redis-py.

[bobbyrward]

Sure, I understand wanting to avoid that. It's just very hard at the moment (read impossible) to use both celery and this library in the same project under python 3.7.

[Grokzen]

Yeah, i see that celery requires >= 3.2.0. I will try to at least push up a update that will put in place the version schema 3.0.x - 3.3.x version support that i plan for the 2.1.0 release. That should at least make it possible to work with this repo in the celery case.

[NeolithEra]

@Grokzen @bobbyrward Thank you for your feedback!

[Grokzen]

Latest commit on master should now allow for any release in the current 3.0.x to 3.3.x version track. Yes building from a git hash is not optimal, but if you want to try it out as a experimental release for now while i update the code to really be compatible with the new redis-py releases anyone can do it now and it should satisfy any librarys current requirements.

[bobbyrward]

Thank for the changes. Master is currently broken unfortunately so we're moving forward with just forking and vendoring redis-py into 2.0.0 for now.

https://github.com/bobbyrward/redis-py-cluster/pull/2