prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.92k stars 161 forks source link

Slow conda resolution #1749

Open aochagavia opened 1 month ago

aochagavia commented 1 month ago

Checks

Reproducible example

pixi init pixi-experiment
cd pixi-experiment
time pixi add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0

Output:

real    2m59.959s
user    3m7.762s
sys     0m28.510s

Issue description

Solving scikit-learn=1.1.1, pandas=1.4.2, pyarrow=5.0.0, matplotlib=3.5.0 takes minutes, which seems too high.

Expected behavior

I'd expect solving to complete much faster, as it does with other requirement combinations (e.g. at least under a minute)

baszalmstra commented 1 month ago

Thanks for the data point! If you have thoughts on how we can improve this Id love to brainstorm!

aochagavia commented 1 month ago

I have no idea why this might be happening, unfortunately... I'll come back and post here if I suddenly have an insight!

synapticarbors commented 1 month ago

Just as a further datapoint, when I use a platform of osx-64, pixi solves in under 40 seconds on a pretty old laptop. However, when I set the platform to linux-64, it grinds away for over 4 minutes. When I run the add command as:

time pixi -vvv  add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0

I can see that resolvo keeps on cycling back to lines like:

INFO resolvo::solver: ╤══ Install pyarrow=5.0.0=py310h225c066_35_cpu at level 50 (required by <root>)

over and over again.

dhirschfeld commented 1 month ago

I was curious:

❯ time pixi add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0
✔ Added scikit-learn=1.1.1
✔ Added pandas=1.4.2
✔ Added pyarrow=5.0.0
✔ Added matplotlib=3.5.0

real    3m28.321s
user    3m56.367s
sys     0m47.349s

It's a bit of an odd environment - pinning an ancient version of pyarrow with more modern deps.