prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
3.21k stars 175 forks source link

PyPI only dependencies should always come from PyPI #2238

Open ruben-arts opened 2 weeks ago

ruben-arts commented 2 weeks ago

Checks

Reproducible example

Setup a project, add and install conda deps, then replace one conda dep with pypi dep.

pixi init test
cd test
pixi add python flask
pixi add --pypi flask
pixi remove flask --no-lockfile-update # Or remove by hand
pixi install

This should have resulted in a pypi package for flask but there are no pypi dependencies found.

❯ pixi list
Package             Version    Build               Size       Kind   Source
...
flask               3.0.3      pyhd8ed1ab_0        78.9 KiB   conda  flask-3.0.3-pyhd8ed1ab_0.conda
...

Issue description

We normally want this behavior if a package is both defined as a conda dependency and a pypi dependency. But since our logic for the satisfiability also checks the locked conda packages against the pypi requirements these packages stay.

The crux is this part of the satisfiability function: https://github.com/prefix-dev/pixi/blob/d50d0d547d04f7de3d69000107e8a9051a6aa495/src/lock_file/satisfiability.rs#L700-L708

Because the (conda_locked_package) identifier satisfies the pypi requirement this function continues and adds the package as a FoundPackage::Conda() which should be only correct if there was already a conda package asking for this.

Expected behavior

I expect the same behavior as when you rm pixi.lock and then pixi install, that flask is a pypi package.

❯ pixi list
Package           Version    Build               Size       Kind   Source
...
flask             3.0.3                          329.2 KiB  pypi   flask-3.0.3-py3-none-any.http.whl
...
skayred commented 1 week ago

Hello,

We are hitting the same issue with Numpy package. Is there any known workaround with the current latest version?

ruben-arts commented 1 week ago

@skayred The trick would be to remove conda version from the lock-file by hand. Which invalidates the lock-file, and should reproduce the correct lock-file. It's a one time thing while you move from conda to pypi so it should reappear.

That said why would you want numpy from pypi? #curious 😄

skayred commented 1 week ago

@ruben-arts thanks, I will try that! For the numpy - it seems that there is some issue with numpy and PyInstaller combo on Windows, although I am not 100% sure it happens because of conda-based numpy