sarugaku / resolvelib

Resolve abstract dependencies into concrete ones
ISC License
138 stars 31 forks source link

Idea: Breadth-first search until reaching breadth level with unpinned requirements #146

Open notatallshaw opened 8 months ago

notatallshaw commented 8 months ago

Due to the iterative nature of resolvelib I don't think can be implemented without the involvement of the provider, and I've not thought about what that API would look like yet, but at the moment the pattern is:

  1. Breadth-first search of top level requirements
  2. Depth-first search of everything else

The idea would be to change it to:

  1. Breadth-first search of top level requirements
  2. If all previous level requirements were pinned (or has only 1 possible version to choose), breadth-first search of the next level
  3. Repeat step 2 until not true
  4. Depth-first search of everything else

There are three common use cases I think this should speed up:

  1. User pins one high level requirement, e.g. apache-airflow[all]==2.7.3, this would breadth-first search one extra level
  2. User pins their own package (my-private-package==1.0), and that package pins all their direct dependencies (dependencies = ['PyYAML==6.0.1', 'requests==2.31.0']), this would breadth-first search two extra levels
  3. User pins their own package (my-private-package==1.0), and provides constraints (-c locked-transitive-dependencies.txt) for all their dependencies and transaitive dependencies, this would breadth-first search all levels