Open notatallshaw opened 4 months ago
One issue could be pip has a preference on what to backtrack on based on certain heuristics of the users requirements, see:
Whereas my understanding is uv does not apply a preference like this.
We added the heuristics in #3087 but this still fails for me, for a different reason related to the relationship between extras and the base package. I have some ideas on how to fix it.
We added the heuristics in #3087 but this still fails for me, for a different reason related to the relationship between extras and the base package. I have some ideas on how to fix it.
Yeah, I was only speculating that pip's heuristics are part of it.
With regards to extras, pip has an optimization that "constrains" extras together with their base requirements: https://github.com/pypa/pip/pull/12095, don't know if that approach will be helpful to you, as no idea how uv handles extras, but for some use cases it was a big performance boost for pip.
Also, my understanding is pubgrub-rs strongly implements CDCL to resolve, whereas I would describe pip's approach as "weakly" implementing CDCL, so the path taken through the resolution graph will always be a little different. Though in this case I would think uv's approach would be better, and I do hope to eventually improve pip here and make it's approach to CDCL stronger.
Unfortunately I still can't get this to avoid going down the apache-beam==2.2.0
branch. I may need to map out the entire path to that requirement to really understand it.
Unfortunately I still can't get this to avoid going down the
apache-beam==2.2.0
branch. I may need to map out the entire path to that requirement to really understand it.
I won't have time today, but I'll try and make a simpler set of requirements that reproduces this issue. apache-airflow[all]
is probably too big to look at the entire path by hand, unless you have exceptional tooling.
Simplifying uv fails with the requirement apache-airflow[apache-beam]==2.8.4
.
When I look at the metadata of apache-airflow
this extra adds the unbound requirement apache-airflow-providers-apache-beam
, new versions of apache-airflow-providers-apache-beam
but a reasonable lower bound on apache-beam
, but older ones do not.
Spending a lot of time playing around with requirements, I found the issue was a combination of dill
and apache-airflow-providers-apache-beam==1.0.0
, simplifying further I found pip is able to resolve this and uv is not:
dill>=0.2.2
apache-beam<=2.49.0
Edit: Constrained failing requirements even further, and deleted the below comment as I wasn't able to reproduce the results this morning and I don't want people to take it as true if I can't reproduce it. Further I want to investigate why pip skips over apache-beam==2.2.0
in several scenarios I tried this morning.
Streps to reproduce
uv pip install --dry-run apache-airflow[all]==2.8.4
Error
Expected
Same as pip, can install fine: