jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.71k stars 611 forks source link

Output --no-binary and --only-binary options to preserve pip behavior #2081

Closed sethmlarson closed 5 months ago

sethmlarson commented 5 months ago

Environment Versions

  1. OS Type: Linux
  2. Python version: 3.12.3
  3. pip version: 24.0
  4. pip-tools version: 7.4.1

Steps to replicate

  1. Compile using `--pip-args='--only-binary=:all: --no-binary=library'
  2. See that compiled requirements list the --no-binary=library option first, --only-binary=:all: option second.
  3. When attempting to install from these requirements the --no-binary is wiped out by the --only-binary=:all:

Expected result

The resulting requirements contain --no-binary and --only-binary options that have the same behavior as input options.

Actual result

Requirements don't have the same behavior as input options.

This improvement matters because using --no-binary/--only-binary is the best way for users to control the amount of potential code execution that happens during the installation process. I have a local fix that I plan on creating a PR.

webknjaz commented 5 months ago

Are you talking about the comment headers in the resulting constraint files?

sethmlarson commented 5 months ago

I'm talking about a file like this one, I discovered this while using --only-binary=:all: for python/release-tools:

#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --generate-hashes --output-file=requirements.txt --pip-args='--only-binary=:all: --no-binary=grapheme,grpclib' requirements.in
#
--no-binary grapheme
--no-binary grpclib
--only-binary :all:

... (all the requirements)

Note the order of --pip-args and that the combination of --no-binary and --only-binary in the resulting requirements.txt don't produce the same behavior in pip.

sethmlarson commented 5 months ago

@webknjaz I've created a PR with a fix that also illustrates the problem: https://github.com/jazzband/pip-tools/pull/2082

webknjaz commented 5 months ago

FWIW I'm of a opinion that the output file should be used as a constraint, not a requirement. As in, pip install -r requirements.in -c requirements.txt. And so such pip options should really go into the input file..