timmc-edx
commented
4 years ago Incidentally, I wrote a script to find things like this across repos I care about: https://github.com/timmc-edx/pip-resolver-compare
Open timmc-edx opened 4 years ago
timmc-edx
commented
4 years ago Incidentally, I wrote a script to find things like this across repos I care about: https://github.com/timmc-edx/pip-resolver-compare
uranusjr
commented
4 years ago So it seems like both resolvers are able to detect that there is a conflict between PyYAML 3.10 and 3.12. The difference is that the current resolver allows that combination anyway, but the new resolver rejects it outright on the basis of the conflict.
What do you think the new resolver should do in this situation, if not immediately rejecting the result? Should it install PyYAML 3.10 or 3.12, and why?
timmc-edx
commented
4 years ago Good question. I'm fairly new to Python dependency management, so I'm not sure I have a well-informed opinion. But for what it's worth... my gut sense is to reject it by default, but allow it (with 3.12, the pinned version) if an appropriate option is set. This is on the premise that someone may very well have a good reason to use conflicting versions—perhaps they're in dependency hell and need to temporarily be in an "invalid" state in order to move through an upgrade process. Something like that.
But I guess I'd first want to know why the current resolver allows it. :-)
uranusjr
commented
4 years ago The current resolver just doesn’t care 😛 It sees 3.12 first, realises it is a viable choice (it hasn’t seen 3.10 yet), and goes “nah, good enough, you know what you’re doing.” The new resolver treats all requirements equally instead, and does not assume the user knows best.
So the problem is two-way. IMO pip can’t just assume the user really knows what they’re doing, since they don’t always do. Maybe the solution for the problem would be to keep the new resolver’s current behaviour, but introduce some kind of an “override” mechanism for those who do know what they’re doing.
I vaguely recall a discussion somewhere on this particular feature, but can’t seem to find the thread now. So I’ll keep this open so people can jump in and provide ideas.
timmc-edx
commented
4 years ago Maybe this is the issue you're thinking of? https://github.com/pypa/pip/issues/8076 "Relaxing / Ignoring constraints during dependency resolution"
timmc-edx
commented
4 years ago (Incidentally, this issue seems to have lost, or never acquired, the "C: new resolver" label.)
uranusjr
commented
4 years ago Ah, yes! Thanks so much for digging it out!
pradyunsg
commented
4 years ago This also came up in #8307, where I'd suggested that such an override mechanism would cover that usecase and others like it.
I found that in a case when the regular resolver would complain about a package incompatibility but install it anyway, the alpha resolver errored out and stopped without installing anything.
It is not clear if this is a bug in the current resolver, the new one, both, or neither. :-)
Inputs
Fresh Python 3.5 virtualenv each time with pip 20.2b1 installed on Ubuntu Linux 18.04.4.
Contents of requirements file:
Old resolver
Output of
pip install -r requirements.txt:(Only portion that was on stderr was the error line above:
ERROR: will 0.9.3 has requirement PyYAML==3.10, but you'll have pyyaml 3.12 which is incompatible.Exit code was zero.)This resulted in the following
pip list --format=freeze:New resolver
Output of
pip install --unstable-feature=resolver -r requirements.txt:Last three lines were on stderr; exit code 1.
This resulted in the following
pip list --format=freeze: