Open brainwane opened 4 years ago
I just wanted to close the loop here from the SkyPortal side. When the new beta resolver was made available, it was unworkable for us. @brainwane reached out, filed this issue, and within a few months our problems were addressed.
A huge shoutout to the team for soliciting community feedback, taking it seriously, and doing such dedicated work to making pip better. I know that many (most?) of you are volunteers, and your efforts are so appreciated. :pray:
Our new dependency resolver may make pip a bit slower than it used to be.
Therefore I believe we need to pull together some extremely rough speed tests and decide what level of speed is acceptable, then build some automated testing to check whether we are meeting those marks.
I just ran a few local tests (on a not-particularly-souped-up laptop) to do a side-by-side comparison:
Or, in 2 different virtualenvs:
These numbers will add up with more complicated processes, dealing with lots of packages at a time.
Related to #6536 and #988.
Edit by @brainwane: As of November 2020 we have defined some speed goals and the new resolver has acceptable performance, so I've switched this issue to be about building automated testing to ensure that we continue to meet our goals in the future.
Edit by @uranusjr: Some explanation for people landing here. The new resolver is generally slower because it checks the dependency between packages more rigorously, and tries to find alternative solutions when dependency specifications do not meet. The legacy resolver, on the other hand, just picks the one specification it likes best without verifying, which of course is faster but also irresponsible.
Feel free to post examples here if the new resolver runs slowly for your project. We are very interested in reviewing all of them to identify possible improvements. When doing so, however, please make sure to also include the
pip install
output, not just yourrequirements.txt
. The output is important for us to identify what pip is spending time for, and suggest workarounds if possible.