search

jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.67k stars 608 forks source link

Option to ONLY install extras from pyproject.toml #2052

Closed jordantshaw closed 6 months ago

jordantshaw commented 7 months ago

What's the problem this feature will solve?

Being able to produce requirement files which only contain the extra dependencies in the pyproject.toml file.

Describe the solution you'd like

For a few of my python applications (not installable libraries) I am using pyproject.toml files to define the top-level dependencies. I like being able to define all of the dependencies within the pyproject.toml file, rather than using requirements.in files.

I would like to be able to create a requirements-dev.txt file based off the optional-dependencies in the pyproject.toml, but only the extra dependencies should be in the requirements-dev.txt file.

webknjaz commented 7 months ago

Extras have a different purpose/semantics. And no installers would install them separately from the mandatory runtime deps. They are exposed to the end-users and are essentially a feature-flag like API. Also, pip-tools is not an installer but a constraint file generator. What you want is defined as "dependency groups" in PEP 735, which is not yet finalized. I mentioned it in passing in my comment about the locking/pinning misconceptions: https://github.com/jazzband/pip-tools/issues/1326#issuecomment-1834517252.

jordantshaw commented 6 months ago

@webknjaz I understand what you are saying. I think I solved my problem using using the constraint arg (-c). Basically I am trying to generate a requirements-dev.txt file, but I want to make sure that the pinned dependencies from the requirements.txt file are used. This does what Im looking for.

pip-compile --upgrade -o requirements/main.txt
pip-compile --upgrade -c requirements/main.txt -o requirements/dev.txt --extra=dev

As per your comment about using extras in a pyproject.toml file to define dev dependencies. I think you are right, that should not be done in a installable library type project. However, in an application project, where code is simply being deployed, I dont see why you shouldn't define dev dependencies in the extra-dependencies section. As there really is no "end-user" of the application.

webknjaz commented 6 months ago

I'd like to emphasize that extras are only defined in the context of distribution metadata. Any other use implies a distribution anyway. To address this separate use case, PEP 735 defines dependency groups that aren't going to end up in the distribution metadata and are tailored specifically to the use case you're describing.

Meanwhile, keep having separate requirements files — they're made for this too, the difference is that they are pip-specific and PEP 735 is going to solve this in a generic cross-compatible tool-agnostic manner.

WhyNotHugo commented 6 months ago

See https://github.com/jazzband/pip-tools/issues/2062 and https://peps.python.org/pep-0735/