prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.19k stars 132 forks source link

Installing PyTorch from default conda command #1544

Open slobodaapl opened 5 days ago

slobodaapl commented 5 days ago

Checks

Reproducible example

pixi global install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

Issue description

Attempting to install PyTorch on Python 3.11 results in failure, due to a package not being detected

..........
would require
0.478   │    └─ blas * mkl, for which no candidates were found.

Expected behavior

Pytorch installs correctly This is likely related to #1484

tibuch commented 1 hour ago

I have the same issue. The following pixi.toml installs only PyTorch CPU:

[project]
name = "pixi-cuda"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim-Oliver Buchholz"]
channels = ["nvidia", "conda-forge", "pytorch"]
platforms = ["linux-64"]

[tasks]
check_cuda = 'python -c "import torch; print(torch.cuda.is_available())"'

[system-requirements]
cuda = "11"

[dependencies]
python = "3.12.*"
pytorch-cuda = ">=11.4,<12"
pytorch = ">=2.3.1,<2.4"

pixi list | grep torch:

libtorch           2.3.1       cpu_mkl_h0bb0d08_100       47.5 MiB   conda  libtorch-2.3.1-cpu_mkl_h0bb0d08_100.conda
pytorch            2.3.1       cpu_mkl_py312h3b258cc_100  23.3 MiB   conda  pytorch-2.3.1-cpu_mkl_py312h3b258cc_100.conda
pytorch-cuda       11.8        h7e8668a_5                 3.5 KiB    conda  pytorch-cuda-11.8-h7e8668a_5.tar.bz2

I can work around it by installing PyTorch via PIP:

[project]
name = "pixi-cuda"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim-Oliver Buchholz"]
channels = ["nvidia", "conda-forge", "pytorch"]
platforms = ["linux-64"]

[tasks]
check_cuda = 'python -c "import torch; print(torch.cuda.is_available())"'

[system-requirements]
cuda = "11"

[dependencies]
python = "3.12.*"

[pypi-dependencies]
torch = "*"

[pypi-options]
extra-index-urls = ["https://download.pytorch.org/whl/cu118"]

pixi list | grep torch:

torch                     2.3.1+cu118                      1.6 GiB    pypi   torch-2.3.1+cu118-cp312-cp312-linux_x86_64.http.whl
tibuch commented 1 hour ago

Okay, the workaround stops working as soon as I add any other dependency like pytorch-lightning:

pixi.toml:

[project]
name = "pixi-cuda"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim-Oliver Buchholz"]
channels = ["nvidia", "conda-forge", "pytorch"]
platforms = ["linux-64"]

[tasks]
check_cuda = 'python -c "import torch; print(torch.cuda.is_available())"'

[system-requirements]
cuda = "11"

[dependencies]
python = "3.12.*"
pytorch-lightning = ">=2.3.0,<2.4"

[pypi-dependencies]
torch = "*"

[pypi-options]
extra-index-urls = ["https://download.pytorch.org/whl/cu118"]

pixi list | grep torch:

libtorch             2.3.1         cpu_mkl_h0bb0d08_100       47.5 MiB   conda  libtorch-2.3.1-cpu_mkl_h0bb0d08_100.conda
pytorch              2.3.1         cpu_mkl_py312h3b258cc_100  23.3 MiB   conda  pytorch-2.3.1-cpu_mkl_py312h3b258cc_100.conda
pytorch-lightning    2.3.0         pyhd8ed1ab_0               432.1 KiB  conda  pytorch-lightning-2.3.0-pyhd8ed1ab_0.conda
torchmetrics         1.4.0.post0   pyhd8ed1ab_0               341.9 KiB  conda  torchmetrics-1.4.0.post0-pyhd8ed1ab_0.conda
tibuch commented 38 minutes ago

And another update. Looks like this is related to the channel order. If I set channels = ["nvidia", "pytorch", "conda-forge"] it works.


[project]
name = "pixi-cuda"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim-Oliver Buchholz"]
channels = ["nvidia", "pytorch", "conda-forge"]
platforms = ["linux-64"]

[tasks]
check_cuda = 'python -c "import torch; print(torch.cuda.is_available())"'

[system-requirements]
cuda = "11"

[dependencies]
python = "3.12.*"
pytorch-lightning = ">=2.3.0,<2.4"
pytorch = ">=2.3.1,<2.4"
pytorch-cuda = "11.8.*"

pixi list | grep torch:

pytorch              2.3.1         py3.12_cuda11.8_cudnn8.7.0_0  1.6 GiB    conda  pytorch-2.3.1-py3.12_cuda11.8_cudnn8.7.0_0.tar.bz2
pytorch-cuda         11.8          h7e8668a_5                    3.5 KiB    conda  pytorch-cuda-11.8-h7e8668a_5.tar.bz2
pytorch-lightning    2.3.0         pyhd8ed1ab_0                  432.1 KiB  conda  pytorch-lightning-2.3.0-pyhd8ed1ab_0.conda
pytorch-mutex        1.0           cuda                          2.8 KiB    conda  pytorch-mutex-1.0-cuda.tar.bz2
torchmetrics         1.4.0.post0   pyhd8ed1ab_0                  341.9 KiB  conda  torchmetrics-1.4.0.post0-pyhd8ed1ab_0.conda