search

pex-tool / pex

A tool for generating .pex (Python EXecutable) files, lock files and venvs.
https://docs.pex-tool.org/
Apache License 2.0
2.53k stars 259 forks source link

Plumb Pip's `--{no,only}-binary`. #2346

Closed jsirois closed 8 months ago

jsirois commented 8 months ago

Previously, Pex just allowed specifying --no-wheel or --no-build globally. Now, individual projects can be slated for --only-wheel or --only-build just as they can with Pip. In the lock case, this information is saved so that lock updates carry the configuration forward.

Fixes #2343

jsirois commented 8 months ago

Ok, @cburroughs I knocked this out and will release 2.1.161 as soon as it lands green here in the hour. My time got cut short this stint and I only have through Saturday evening to bang out #2344. I think I'll get that done, but in case not, this should unblock your Pants work on the sharper knife path if you don't want to potentially wait until ~February 9th when I next pick back up work after going AFK this Sunday.

cburroughs commented 8 months ago

Thank for the update, context, and enjoy your trip! I'll try to give it a look right now.

It lookslike the pex has an extra validation that stops the pip special :all: and :none values from flowing through:

$ python3.10 -m pex.cli lock create '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac --only-binary :all:  ansicolors==1.1.5 kaleido==0.1.0.post1   -o tmp/k2.json 
usage: /usr/bin/python3.10 -m pex.cli lock create [-h] [--style {strict,sources,universal}] [--target-system {linux,mac,windows}]
                                                  [--path-mapping PATH_MAPPINGS] [-o PATH] [--indent INDENT] [-r FILE or URL]
                                                  [--constraints FILE or URL] [--python PYTHON] [--python-path PYTHON_PATH]
                                                  [--interpreter-constraint INTERPRETER_CONSTRAINT] [--platform PLATFORMS]
                                                  [--complete-platform COMPLETE_PLATFORMS] [--manylinux [ASSUME_MANYLINUX]]
                                                  [--resolve-local-platforms]
                                                  [--resolver-version {pip-legacy-resolver,pip-2020-resolver}]
                                                  [--pip-version {latest,vendored,20.3.4-patched,22.2.2,22.3,22.3.1,23.0,23.0.1,23.1,23.1.1,23.1.2,23.2,23.3.1,23.3.2}]
                                                  [--allow-pip-version-fallback] [--use-pip-config] [--pypi] [-f PATH/URL]
                                                  [-i URL] [--retries RETRIES] [--timeout SECS] [--proxy PROXY] [--cert PATH]
                                                  [--client-cert PATH] [--cache-ttl DEPRECATED] [-H DEPRECATED] [--pre] [--wheel]
                                                  [--only-binary ONLY_WHEELS] [--build] [--only-build ONLY_BUILDS]
                                                  [--prefer-wheel] [--force-pep517] [--build-isolation] [--transitive] [-j JOBS]
                                                  [--preserve-pip-download-log] [-v] [--emit-warnings] [--pex-root PEX_ROOT]
                                                  [--disable-cache] [--cache-dir CACHE_DIR] [--tmpdir TMPDIR] [--rcfile RC_FILE]
                                                  [requirements ...]
/usr/bin/python3.10 -m pex.cli lock create: error: argument --only-binary/--only-wheel: The given project name ':all:' is not a valid. It must conform to the regex '^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$' as specified in https://peps.python.org/pep-0508/#names
jsirois commented 8 months ago

Sorry about that! It looks like I edited your comment instead of replying. Very bad.

It looks like the pex has an extra validation that stops the pip special :all: and :none values from flowing through:

Correct! That's what the pre-existing --no-{wheel,build} options are for. In other words these are not direct pass throughs. They only logically pass through. See here for the exact mapping step setting up Pip's args: https://github.com/pantsbuild/pex/pull/2346/files#diff-443e54ea41451726af072f7976469d8bdbb37d285fd49fc1a4bc6447588591c3R409

For context, the --no-{wheel,build} options were introduced back in Pex "OLD" (2014) when it had a bespoke resolver and did not use Pip. Since Pex doesn't break existing users, the option names and meanings stayed constant after adoption of Pip as the underlying resolver in the 2.x series.