uranusjr
commented
3 years ago The algorithms (both new and old) guarentee to return a solution that meets all user requests, but never promises it would be “the” result. In practice, the result returned by either resolver is deterministic, otherwise it would be extremely difficult to debug them otherwise, but there is no guarantee it would always match exactly what you want, unless you tell it what you want.
In other words, this is expected, and I don’t plan to change it because the behaviour is IMO correct. If you need specific versions of packages, you should tell pip about it.
brainwane
joestringer
What did you want to do?
At a high level, build the documentation for the Cilium project. We observed that automated builds began to fail around or after 30 November 2020.
More specifically, use pip to manage dependencies where we have a package that is provided differently depending on the architecture.
Input
requirements.txt:
Usage
On earlier pip versions, when running on non-aarch64 systems, this would silently resolve the conflict to install the forked version of the package from git. On aarch64 systems, the package would be provided by the upstream sphinx-rtd-theme package.
Output
After upgrading to pip 20.3, the conflict was silently resolved in the alternate order which subsequently lead to build problems due to the dependency being resolved differently.
Additional information
We use pip for this purpose in two ways:
Workarounds:
--use-deprecated=legacy-resolveron the pip commandlineFor readthedocs, neither of these options are available as we are not in control of the pip commandline, only the requirements.txt file.
Having looked at the issue for a while now, I believe that ultimately the cause of this issue stems from the ambiguity in the
requirements.txtfile, so by clarifying the conflict in the requirements file, we were able to mitigate the issue (https://github.com/cilium/cilium/pull/14264). We were not previously aware that therequirements.txtwas ambiguous, or it did not seem to matter as the end result (build the docs successfully) was achieved.Expected behavior from pip
I would have preferred if pip rejected my combination of requirements, informing me that there is a conflict, and providing me with a message that tells me that this is the problem and I can either resolve it (ideally giving some hints as to what the problem is, at least which packages are involved in the conflict) or alternatively, specify some override flag to tell pip "I don't care, figure it out whichever way you want and I will deal with the consequences" (which would have the same behaviour as pip 20.3 today).
At this point I tried to figure out if there are any requirements.txt linters or --strict-mode or anything that I could enable to detect such problems and inform me in advance, but so far I was unable to locate any such tools.