Is `pip install -r a.txt -r b.txt` equivalent to `pip install -r b.txt -r a.txt`?

[harupy]

Description

This is not an issue, but a question. Is pip install -r a.txt -r b.txt equivalent to pip install -r b.txt -r a.txt?

Expected behavior

No response

pip version

-

Python version

-

OS

-

How to Reproduce

-

Output

No response

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[harupy]

In our repo (mlflow/mlflow), we run a command that looks similar to pip install -r a.txt -r b.txt. We found pip install -r a.txt -r b.txt ends up hanging but pip install -r b.txt -r a.txt works.

[pfmoore]

They are logically equivalent, but the order in which requirements are presented to pip can affect the backtracker's route through the dependency graph. This can mean, in complex cases, that ordering can be the difference between long resolve times (which often look like hanging) and quick ones. You can get this by swapping the order of the requirements files, or by reordering requirements within those files.

So the answer is "yes, they are equivalent, up to a point" 🙂

[harupy]

@pfmoore Thanks for the quick answer! Understood :)

[notatallshaw]

In our repo (mlflow/mlflow), we run a command that looks similar to pip install -r a.txt -r b.txt. We found pip install -r a.txt -r b.txt ends up hanging but pip install -r b.txt -r a.txt works.

Btw do you have a reproducible example?

Two big improvements in https://github.com/sarugaku/resolvelib have happened, one has landed on Pip main and another has landed on resolvelib main but not Pip main yet, I'd be happy to check if this issue is resolved by either of these two improvements.

I have yet to find an example where Pip significantly hangs anymore from backtracking with both improvements applied 😃 .

[notatallshaw]

They are logically equivalent, but the order in which requirements are presented to pip can affect the backtracker's route through the dependency graph.

Just to expand on this a little bit, Pip finds the first solution it can for the given requirements, and this will probably always have to depend on user order.

Let's say you had the very simple requirements of A and B, each have two releases of the following dependencies:

B=2 depends on A < 2
B=1 has no dependencies
A=2 depends on B < 2
A=1 has no dependencies

If you specify the requirements A B then A=2 is downloaded first, the requirement B < 2 is added, so the next requirement B is downloaded with this additional requirement so B=1 is downloaded. A=2 and B=1 matches the requirement so they are installed.

On the other hand if you specify the requirements B A then B=2 is downloaded first, the requirement A < 2 is added, so the next requirement A is downloaded with this additional requirement so A=1 is downloaded. A=1 and B=2 matches the requirement so they are installed.

Hope that helps make some more sense of why user order can cause weird behavior.