Allow to use --only-binary=:all: but exclude a few packages.

Carreau commented 1 year ago

What's the problem this feature will solve?

I like to have --only-binaries=:all:, especially in CI, but not all packages are availlable as binary everywhere (especially since 3.12 is recent).

I'd like to say "all binary, except for X, Y, Z". doing --only-binaries=:all: --no-binary=X,Y,Z does not cut it as now on older Python where binary is available it will still use the TGZ.

Describe the solution you'd like

Either a flag --binary-or-source=X,Y,Z, or a syntax to say --only-binary=:all:,-X,-Y,-Z

Note that this is different from --prefer-binary.

Alternative Solutions

List all packages instead of :all:.

It is impractical because of the potential CLI length, and recursive dependencies that may change.

Ask upstream to publish wheels (not always possible)
Install in several steps with combinaison of flags (not always possible).

Additional context

I believe that should not be too invasive of a change as most changes should be on the FormatControl class if my understanding is correct.

Code of Conduct

[X] I agree to follow the PSF Code of Conduct.

pfmoore commented 1 year ago

I have hit similar issues, so I'm in favour of the idea in principle. However, I think our options in this area are already rather complex and confusing (the differences and interactions between --only-binary, --prefer-binary, and --no-binary are far from obvious). I think we should be cautious about simply adding new options here, and maybe instead we should work out some form of replacement syntax that allows the user to describe clearly what they want.

I'm conscious though that "the perfect is the enemy of the good" and "practicality beats purity", so I don't want to get side-tracked designing some sort of selection language rather than just fixing the issue - but I do think we should at least consider a more general mechanism, rather than just adding more technical debt.

Also relevant here is the discussion in #9140.

Carreau commented 1 year ago

Thanks, I agree but took a conservative stance in my issue. I think a mini language say +foo -foo ~foo in addition to :all: :none: could maybe work. It could be an exclusive flag with a long deprecation period.

Also if we look at wider problem, should this be extends to requirements file (i'm inclined to say no) or just pip cli ?

I also have the vague impression that we might be able to solve as well without adding new flags. as :none: empties the current set we could have --no-binary foo,:none:,bar "remove" foo from the only binary set. I don't like it, but it does not add any external things and rely on the fact that if something is both only binary and no binary, you will never be able to find a viable source.

On Fri, Oct 13, 2023 at 17:24 Paul Moore @.***> wrote:

I have hit similar issues, so I'm in favour of the idea in principle. However, I think our options in this area are already rather complex and confusing (the differences and interactions between --only-binary, --prefer-binary, and --no-binary are far from obvious). I think we should be cautious about simply adding new options here, and maybe instead we should work out some form of replacement syntax that allows the user to describe clearly what they want.

I'm conscious though that "the perfect is the enemy of the good" and "practicality beats purity", so I don't want to get side-tracked designing some sort of selection language rather than just fixing the issue - but I do think we should at least consider a more general mechanism, rather than just adding more technical debt.

Also relevant here is the discussion in #9140 https://github.com/pypa/pip/issues/9140.

— Reply to this email directly, view it on GitHub https://github.com/pypa/pip/issues/12348#issuecomment-1761700992, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACR5T4NPEQMQHXYOZZUVQLX7FML3AVCNFSM6AAAAAA57DDIDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRRG4YDAOJZGI . You are receiving this because you authored the thread.Message ID: @.***>

zerothi commented 3 months ago

This would be great!!!

I also kind of like:

pip install --only-binary=':all:,^package'

to exclude package from the :all: set. I think having the - check is not clear enough... Could be wrong here on the popular opinion.

alexlancaster commented 1 week ago

What's the problem this feature will solve?

I like to have --only-binaries=:all:, especially in CI, but not all packages are availlable as binary everywhere (especially since 3.12 is recent).

I'd like to say "all binary, except for X, Y, Z". doing --only-binaries=:all: --no-binary=X,Y,Z does not cut it as now on older Python where binary is available it will still use the TGZ.

hi there, I just filed #13077, because, at least for me using the --only-binary=:all: --no-binary=docopt approach does seem to work for me OK (I want to exclude docopt), from the command-line, but it doesn't work for the environment variable equivalents (PIP_ONLY_BINARY / PIP_NO_BINARY). Does this not work for you even for the command-line version?

Irrespective of the current behaviour, I agree with the other suggestions here for a more intuitive way to do this would be good, especially because of the cibuildwheel case, where we really want to ensure that the only wheels that are built, are ones that do not need compilation by the end user

pypa / pip