Add `--only-deps` (and `--only-build-deps`) option(s)

[flying-sheep]

https://github.com/pypa/pip/issues/11440#issuecomment-1445119899 is the currently agreed upon user-facing design for this feature.

---

What's the problem this feature will solve?

In #8049, we identified an use case for installing just the dependencies from pyproject.toml.

As described in the solution section below --only-deps=<spec> would determine all dependencies of <spec> excluding that package itself and install those without installing the package. It could be used to

1. allow specifying environment variables that are active only while building a package of interest (without having it be active while potentially building its dependencies).
2. separate dependency installation from building and installing a package, allowing to rebuild a package in a docker build while the dependency installation step is loaded from cache.

This example shows both use cases:

# copy project metadata and install (only) dependencies
COPY pyproject.toml /myproj/
WORKDIR /myproj/
RUN pip install --extra-index-url="$PIP_INDEX" --only-deps=.[floob]

# copy project source files, build in a controlled environment and install our package
COPY src/mypkg/ /myproj/src/mypkg/
RUN env SETUPTOOLS_SCM_PRETEND_VERSION=2.0.2 python3 -m build --no-isolation --wheel
RUN pip install --no-cache-dir --no-dependencies dist/*.whl

Instead of the solution from #8049, @pradyunsg prefers a solution similar to the one below: https://github.com/pypa/pip/issues/8049#issuecomment-1079882786

Describe the solution you'd like

One of those two, or similar:

1. (used in the example above)

   --only-deps would work like -r in that it’s not a flag globally modifying pip’s behavior but a CLI option with one argument that can be specified multiple times. Unlike -r it accepts a dependency spec and not a path to a file containing dependency specs.

   Where pip install <spec> first installs all dependencies and then (build and) install the package referred to by the spec itself, pip install --only-deps=<spec> would only install the dependencies.

2. --only-deps would work like --[no|only]-binary, in that it requires an argument specifying what package not to install. A placeholder like :requested: could be used, e.g.:

   pip install --only-deps=:requested: .[floob]

Alternative Solutions

• Re-using -r instead of adding --only-deps.

  I don’t think this is a good idea, since people would be tempted to do -r pyproject.toml which would be wrong (Dependency specs including file paths look like like ./path/to/pkg[extra1,extra2])

• Making --only-deps a global switch modifying pip’s behavior like e.g. --pre.

  I have found that global switches like that are dangerous and not very intuitive. To install a dev version of your package, doing pip install --pre mypkg seems innocuous but will actually install dev versions of mypkg and all its dependencies that have any dev versions. It’s safer to do something like pip install mypkg>=0.1.post0.dev0 to limit dev version installations to one package. Similarly it’s unclear what a --only-deps switch would apply to. Would pip install -r reqs.txt --only-deps install the dependencies of every package specified in the file but none of those packages?

• Using e.g. beni to convert PEP 621 dependencies to a requirements.txt.

  This works even today but feels like is shouldn’t be necessary as it involves quite a few steps, including writing a file to disk.

Additional context

NA

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[MohammadRaziei]

I really need this feature. Is it planned for the future?

[stefanv]

In scikit-image, we've resorted to writing out various requirement.txt files each time pyproject.toml is updated.

[webknjaz]

PSA: It will be possible to extract the build deps (with their transitive requirements) of a project in the next pip-tools release — 3 weeks ago we merged a PR that took like a year to polish: https://github.com/jazzband/pip-tools/pull/1681.

@stefanv I also used to maintain a manually extracted copy of the build deps for use with PIP_CONSTRAINT in the past, but the new feature is going to cover that need. If you'd like to try it out, test the version from Git. See https://pip-tools.rtfd.io/en/stable/#maximizing-reproducibility for the hints.

@pfmoore this new pip-tools feature extracts both static and dynamic deps using said callbacks FYI.

P.S. As for the --only-deps feature — I think it's a can of worms, this would encourage projects to expose things through metadata that are meant to be private. Like, why would an end user (not a contributor) need access to the docs or test deps? Those deps usually need env-dependent pinned constraints rather than follow the normal pkg runtime dep considerations… Ah, it looks like PEP 735 is attempting to address this.

[stefanv]

PSA: It will be possible to extract the build deps (with their transitive requirements) of a project in the next pip-tools release — 3 weeks ago we merged a PR that took like a year to polish: jazzband/pip-tools#1681.

From reading that PR, it looks like you will have to compile the package in order to extract the build dependencies?

[webknjaz]

@stefanv no, you can feed it a pyproject.toml. The PEP 517 hooks are called to augment the statically listed build deps. But then, there's also dependency resolution to get the transitive deps, so that might end up building some of the packages in the tree, if that's what you're asking.

[evbo]

Python developers for years if not decades have followed a very simple deploy process for standalone apps:

1. create a venv (virtual environment) and source ./venv/bin/activate
2. pip install - r
3. run the code as-is, package it with pyinstaller, or build a wheel as needed...

For reverse compatibility and to unify the Python development experience, as part of this PR could you please add to pyproject.toml a [build-system] setting that disables every other step besides install dependencies? Then we get an even simpler command (pip install .) and identical behavior exactly like pip install -r provided. It's important this be codified inside pyproject.toml so that it's self-documenting how to build the code (without redundantly coding it for CI automation with --only-deps arguments, or simply without having to remember when installing manually, etc.).

tl;dr: adding a only_deps flag to [build-system] is valuable for 3 reasons:

1. complete backwards compatibility with pip install -r into a venv deployment flow described above
2. encourages migration of legacy projects to completely co-locate everything in pyproject.toml
   • Like Rust/Scala/NPM and so many other build systems for other languages
3. self-documenting pyproject.toml
   • No extra coding in CI automation using lesser known --only-deps flags or side-effecting pip-compile

Similar sentiments expressed: https://stackoverflow.com/questions/74354009/install-dependencies-from-pyproject-toml-but-not-the-package https://stackoverflow.com/questions/77150911/equivalent-to-pip-install-r-requirements-dev-txt-for-pyproject-toml https://github.com/astral-sh/uv/issues/1516 https://github.com/pypa/flit/pull/631 https://github.com/pypa/pip/issues/12425 https://github.com/pypa/pip/issues/2171 https://github.com/pypa/pip/issues/11584

[webknjaz]

PSA: It will be possible to extract the build deps (with their transitive requirements) of a project in the next pip-tools release — 3 weeks ago we merged a PR that took like a year to polish: jazzband/pip-tools#1681.

FTR that feature has finally been released in pip-tools v7.4.0. So the build deps constraints/lockfile are extractable using something like pip-compile --all-build-deps --output-file=build-constraints.txt --strip-extras pyproject.toml now. Hopefully, this will provide a workaround for a portion of this feature request.

From reading that PR, it looks like you will have to compile the package in order to extract the build dependencies?

@stefanv I only now realized why you were asking about compiling :laughing: The command pip-compile is a pip-tools CLI for making constraint files out of input deps, that's what "compiling" means in that context.

[matterhorn103]

Edit: this was just a plea to remember the science use-case, but it added nothing to the discussion in the light of @webknjaz's reply. Sorry!

[webknjaz]

@matterhorn103 pretty sure that it's pending the acceptance/completion of PEP 735 so that there could be an installer-agnostic way of declaring those. It specifically mentions the scientific projects among the use cases, by the way. Please, check it out, it's an interesting read.

[matterhorn103]

@webknjaz Ah, I should have known better than to comment on such an old thread... Thanks for the reply and the pointers. I also since then found threads like this one discussing the future options and particularly the scientific context.

[charliermarsh]

There are some references in here to reading metadata from pyproject.toml directly. We do this in uv (when the dependencies are not declared as dynamic, etc.), so just to share from my own experience, the one issue we've run into is that some tools have added non-standard extensions to the project.dependencies table.

For example, in Hatch, you do:

[project]
dependencies = [
  "black @ {root:uri}/black_editable"
]

...to declare a dependency at a path relative to the current project. This is called the Context formatting API.

Similarly, PDM injects a magic ${PROJECT_ROOT} variable for local dependencies:

 [project]
dependencies = [
    "sub-package @ file:///${PROJECT_ROOT}/sub-package",
    "first @ file:///${PROJECT_ROOT}/first-1.0.0-py2.py3-none-any.whl",
]

I think we actually support that in uv too (it falls out of supporting environment variable expansion in requirements.txt parsing), though I'd like to get rid of it. I view all of these as quite clearly violating PEP 621 so pip would need to make a decision as to whether they're supported, if it were to start reading from pyproject.toml directly.

[sbidoul]

If I remember correctly, the goal of static/dynamic in PEP 621 was to allow for backend interoperability and allow decision making on the metadata without invoking the backend.

I view all of these as quite clearly violating PEP 621

So I share this view, as they prevent these optimizations.

Actually, I think frontends should check that backends honor static metadata in the wheels they build.

[brettcannon]

I view all of these as quite clearly violating PEP 621

So do I.