Installation fails due to conflicting pyyaml version

[NeolithEra]

Hi, users are unable to run Zelt due to dependency conflict with pyyaml package. As shown in the following full dependency graph of Zelt, Zelt requires pyyaml>=5.1,<6.0，while kubernetes>=10.0.1,<11.0.0 requires pyyaml~=3.12.

According to pip’s “first found wins” installation strategy, pyyaml 5.3.1 is the actually installed version. However, pyyaml 5.3.1 does not satisfy pyyaml~=3.12.

Dependency tree-----------

zelt - 1.2.14
| +- boto3(install version:1.13.18 version range:>=1.9,<2.0)
| +- docopt(install version:0.6.2 version range:>=0.6.2,<0.7.0)
| +- greenlet(install version:0.4.15 version range:>=0.4.15,<0.5.0)
| +- har-transformer(install version:1.2.6 version range:>=1.0,<2.0)
| | +- chevron(install version:0.13.1 version range:>=0.13,<0.14)
| | +- dataclasses(install version:0.6 version range:>=0.6.0,<0.7.0)
| | +- docopt(install version:0.6.2 version range:>=0.6.2,<0.7.0)
| | +- ecological(install version:1.6.0 version range:>=1.6,<2.0)
| | +- pendulum(install version:2.1.0 version range:>=2.0,<3.0)
| | | +- python-dateutil(install version:2.8.1 version range:>=2.6,<3.0)
| | | +- pytzdata(install version:2019.3 version range:>=2018.3)
| | +- requests(install version:2.23.0 version range:>=2.21,<3.0)
| | | +- certifi(install version:2020.4.5.1 version range:>=2017.4.17)
| | | +- chardet(install version:3.0.4 version range:>=3.0.2,<4)
| | | +- idna(install version:2.9 version range:>=2.5,<3)
| | | +- urllib3(install version:1.25.9 version range:>=1.21.1,<1.26)
| +- kubernetes(install version:10.1.0 version range:>=10.0.1,<11.0.0)
| | +- certifi(install version:2020.4.5.1 version range:>=14.05.14)
| | +- google-auth(install version:1.14.0 version range:>=1.0.1)
| | | +- cachetools(install version:4.1.0 version range:>=2.0.0,<5.0)
| | | +- pyasn1-modules(install version:0.2.8 version range:>=0.2.1)
| | | +- rsa(install version:4.0 version range:>=3.1.4,<4.1)
| | | | +- pyasn1(install version:0.4.8 version range:>=0.1.3)
| | | +- setuptools(install version:46.1.3 version range:>=40.3.0)
| | | +- six(install version:1.14.0 version range:>=1.9.0)
| | +- python-dateutil(install version:2.8.1 version range:>=2.5.3)
| | +- pyyaml(install version:3.13 version range:<4,>=3.12)
| | +- requests(install version:2.23.0 version range:*)
| | | +- certifi(install version:2020.4.5.1 version range:>=2017.4.17)
| | | +- chardet(install version:3.0.4 version range:>=3.0.2,<4)
| | | +- idna(install version:2.9 version range:>=2.5,<3)
| | | +- urllib3(install version:1.25.9 version range:>=1.21.1,<1.26)
| | +- requests-oauthlib(install version:1.3.0 version range:*)
| | +- setuptools(install version:46.1.3 version range:>=21.0.0)
| | +- six(install version:1.14.0 version range:>=1.9.0)
| | +- urllib3(install version:1.25.8 version range:>=1.24.2)
| | +- websocket-client(install version:0.57.0 version range:>=0.32.0)
| | | +- six(install version:1.14.0 version range:*)
| +- locustio(install version:0.9.0 version range:>=0.9.0,<0.10.0)
| | +- flask(install version:1.1.2 version range:>=0.10.1)
| | | +- click(install version:7.1.1 version range:>=5.1)
| | | +- itsdangerous(install version:1.1.0 version range:>=0.24)
| | | +- Jinja2(install version:2.11.2 version range:>=2.10.1)
| | | | +- MarkupSafe(install version:2.0.0a1 version range:>=0.23)
| | | +- Werkzeug(install version:1.0.1 version range:>=0.15)
| | +- gevent(install version:1.5.0 version range:>=1.2.2)
| | +- msgpack(install version:1.0.0 version range:>=0.4.2)
| | +- pyzmq(install version:19.0.0 version range:>=16.0.2)
| | +- requests(install version:2.23.0 version range:>=2.9.1)
| | | +- certifi(install version:2020.4.5.1 version range:>=2017.4.17)
| | | +- chardet(install version:3.0.4 version range:>=3.0.2,<4)
| | | +- idna(install version:2.9 version range:>=2.5,<3)
| | | +- urllib3(install version:1.25.9 version range:>=1.21.1,<1.26)
| | +- six(install version:1.14.0 version range:>=1.10.0)
| +- pyyaml(install version:5.3.1 version range:>=5.1,<6.0)
| +- tenacity(install version:5.1.5 version range:>=5.0,<6.0)

Thanks for your help. Best, Neolith

[NeolithEra]

Solution

1. Loosen the version range of pyyaml to be <=4.0.
2. Remove your direct dependency pyyaml, and use the pyyaml transitively introduced by kubernetes.
3. Ask your direct dependency kubernetes to loosen the version range of pyyaml to be >=5.1,<6.0. @bmaher Which solution do you prefer, 1 ,2 or 3? Please let me know your choice. May I pull a request to solve this issue?

[thilp]

Hi @NeolithEra! It looks like kubernetes always used pyyaml>=3.12 (e.g. in v10.0.1 or v11.0.0 or master), except for v10.1.0 where they used pyyaml~=3.12 for some reason. That's clearly not a "non breaking change" according to semver, but :man_shrugging:

In my opinion the simplest fix on our side is to update our bounds on kubernetes:

@@ -23,7 +23,7 @@ packages = [{include = "zelt"}, { include = "main.py"}]
 [tool.poetry.dependencies]
 python = "^3.6"
 har-transformer = "^1.0"
-kubernetes = "^10.0.1"
+kubernetes = "^11.0.0"
 locustio = "^0.9.0"
 boto3 = "^1.9"
 greenlet = "^0.4.15"

You're of course welcome to propose a pull request, otherwise without any other opinion I'll go ahead and make the update myself this evening. What do you think?

[tortila]

Interesting to see that the reason for this change in v10.1.0 was that pyyaml stopped supporting Python 3.4: https://github.com/kubernetes-client/python/pull/1079#issuecomment-589816780. Kubernetes client also drops that support in v11.0.0a1