Closed tstadel closed 2 years ago
Consolidating into https://github.com/pypa/pip/issues/9254.
I appreciate that complicated dependency graph problems are take a non-neglibile amount of work to detangle and not presenting information about why pip is backtracking can make it dififcult to understand. That said, the specific request for more information is already being tracked in #9254. There have been discussions about "freshness" and "breadth first" search reordering, on this issue tracker -- which can be summarised as, they might seem to be an improvement, but (a) it'd likely end up being more difficult to reason about + control for an end user and (b) the information is not as useful for heuristics as it might seem.
Description
pip gets completely lost during backtracking when installing
farm-haystack[dev]==1.2.0
. Depending on which compiler tooling you have installed one of the following will happen:This started about a week after releasing version 1.2.0 of
farm-haystack
because a subdependency namedazure-core
got a new version (1.23) which resulted in a overall dependency conflict of packagetyping-extensions
.Analyzing this we had a very bad experience. Currently in pip there is no way to see which conflict causes this problem. Only digging through pip issues and finding the branch of https://github.com/pypa/pip/pull/10258 helped us solve this. The output of pip on this branch showed us:
Further analyzing with pipdeptree showed us which direct dependencies of farm-haystack caused this conflict:
So we have two direct dependencies
pydoc-markdown
andazure-ai-formrecognizer
which have different subdependenciesdatabind.core
andazure-core
which itself need a conflicting version oftyping-extensions
. Version 1.23.0 of azure-core introduced the dependency oftyping-extensions>=4.0.1
and thus the conflict.Solving this would be as easy as restricting
azure-core<1.23
but analyzing without the branch would have taken hours if not days.See respecting haystack issue.
Expected behavior
At least there should be transparency about which conflict started backtracking like in https://github.com/pypa/pip/pull/10258 or https://github.com/pypa/pip/pull/10937. Additionally the backtracking approach of depth-first search seems to be suboptimal in certain situations. Given this situation in which an unpinned subdependency releases a new version which introduces a new dependency conflict might not be too uncommon. So I wonder whether breath-first search would be more suitable or freshness of package versions in general should have an impact on deciding where to backtrack next.
pip version
22.0.4
Python version
">=3.7"
OS
ubuntu, windows, (any?)
How to Reproduce
Try https://colab.research.google.com/drive/1UtUDxBRZcaGLMFs7Nh23JRXl_f_50VwW?usp=sharing or
Output
Checkout https://colab.research.google.com/drive/1UtUDxBRZcaGLMFs7Nh23JRXl_f_50VwW?usp=sharing for full output (it still seems to be backtracking) or
Code of Conduct