astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
28.31k stars 808 forks source link

Explicit index not respected for transitive dependency #8253

Open juzna opened 1 month ago

juzna commented 1 month ago

Hey, thanks for delivering explicit indexes in #7481 ❤️ I'm trying this with uv built from the last commit (e71b1d0c), but I have a problem with transitive dependencies & explicit index.

Scenario: my-app depends on internal pkg a, which declares a dependency on (internal) package b.

In my app project.toml, I define the index plus I set sources for both pkg a & b. But thenuv lock` gives me

Because b was not found in the package registry ...

So it's ignoring the source definition when resolving a transitive dependency.

[[tool.uv.index]]
name = "my-internal-index"
url = "..."
explicit = true

[tool.uv.sources]
a = { index = "my-internal-index" }
b = { index = "my-internal-index" }

[project.dependencies]
a = { version = "1.0" }  # `a` declares a dependency on `b`
charliermarsh commented 1 month ago

Sources only apply to dependencies that are declared in the project itself. In this case, you'd need to define b as a dependency.

juzna commented 1 month ago

Yep, that works.

Though, is it a good approach? my-app doesn't really depend on b, only transitively. So if a removes the dependency on b in the future, I'll be left with an unused dependency. For this reason, I'll also get a warning by deptry that the dependency is not used in my code.

corleyma commented 1 month ago

I don't know if it applies here, but this handling of transitive dependencies is the primary driver for the "glob pattern-based source declaration" functionality that exists in PDM and others have implemented using poetry plugins described e.g. here. The idea is that in corporate environments, where it's common to enforce a namespace for internal packages, this makes it easy to setup a declaration that handles pinning transitive internal dependencies to a particular repository.

just echoing that there is some desire to have a way to inform a package manager about how to handle particular transitive dependencies.

odie5533 commented 1 month ago

@juzna You may be better off though removing the explicit = true so that all packages are checked on your internal index first. That's probably a fair workaround, and improves your security.

mnchsmn commented 1 month ago

We're also running into this issue. We have a library that depends on torch, so for that library we have set up as in the docs, and that works.

[tool.uv.sources]
torch = { index = "pytorch" }

[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

Now when this library is used in an application, I'm getting Because there is no version of torch{platform_system == 'Linux'}==2.5.0+cpu..., even when the same index is also added in the application's pyproject.toml. Making the index non-explicit also doesn't work, because the torch index contains some older version of requests which then blocks us from fetching the required requests from the default index. It does work with non-explicit index and --index-strategy unsafe-first-match but we'd prefer to avoid using the unsafe option.