astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
27.17k stars 785 forks source link

Packages depending on torch do not respect torch version #9306

Closed mbignotti closed 1 day ago

mbignotti commented 1 day ago

Hi! First of all, I wanted to thank you for the new pytorch packaging guide and for the project as whole. It's great!

I'm having an issue in my current project, where I'm trying keep the cpu and gpu torch versions separate, as in your example. I need to install an additional package, ultralytics, that depends on pytorch, and I would like it to respect the selected torch version. Here is my base setup:

[project]
name = "test"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
authors = [
    { name = "mbignotti", email = "marco.bignotti@it.abb.com" }
]
requires-python = ">=3.10"
dependencies = [
    "polars>=1.14.0",
]

[project.scripts]
test = "test:main"

[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project.optional-dependencies]
cpu = [
  "torch>=2.5.1",
  "torchvision>=0.20.1",
  "ultralytics>=8.3.34",
]
cu124 = [
  "torch>=2.5.1",
  "torchvision>=0.20.1",
  "ultralytics>=8.3.34",
]

[tool.uv]
conflicts = [
  [
    { extra = "cpu" },
    { extra = "cu124" },
  ],
]

[tool.uv.sources]
torch = [
  { index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
  { index = "pytorch-cu124", extra = "cu124" },
]
torchvision = [
  { index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
  { index = "pytorch-cu124", extra = "cu124" },
]

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "pytorch-cu124"
url = "https://download.pytorch.org/whl/cu124"
explicit = true

However, when I run uv sync --extra cpu, the gpu torch version is downloaded anyways since ultralytics depends on torch, and on Linux the default pytorch version is the gpu one. I've already tried to move the ultralytics dependency in a separate optional group or in a dependency group, but nothing changes even when I install it after installing torch. If I use the pip interface, instead, it works. The two following commands respect the specified torch version:

uv sync --extra cpu
source .venv/bin/activate && uv pip install ultralytics 

However, packages installed with the pip interface are not tracked in the pyproject.toml and in the lock file.

Do you have any idea about how to solve this problem? Am I doing something wrong?

Thanks a lot!

charliermarsh commented 1 day ago

I believe this is actually the same as #9289, so let's track there. Thanks for the clear write-up.

charliermarsh commented 1 day ago

Definitely a bug though. I believe the general issue lies in having dependencies that depend on a package that's listed multiple times, across conflicting groups.