search

pex-tool / pex

A tool for generating .pex (Python EXecutable) files, lock files and venvs.
https://docs.pex-tool.org/
Apache License 2.0
2.5k stars 257 forks source link

Implement pex3 lock sync. #2373

Closed jsirois closed 4 months ago

jsirois commented 5 months ago

Introduce the high level pex3 lock sync command. This should generally suffice for typical use cases and works as follows:

In addition to creating and syncing a lock, it can also create and sync a venv (--venv) based on the lock. Further, a command can be specified to run in the synchronized venv with arguments following the -- option terminator.

This latter set of features allow Pex to act as a concise tool in tox / nox / invoke / make setups to implement a simple build system.

Fixes #2344

jsirois commented 5 months ago

Another pretty huge one. Thanks in advance for any scrutiny you can provide.

I did not convert Pex's tox setup to use itself as the venv provisioner in this PR to keep the already large diff size down. There will be a follow up for that though as well as similar use in a-scie/lift (science) where Pex is already heavily used, but in a cumbersome way, to drive the underlying nox venvs.

jsirois commented 5 months ago

@cburroughs I sent you an invite. I'd like to have you review, especially with an eye towards making sure you have everything you need for your Pants work with this change.

cburroughs commented 5 months ago

Thank you! I'll take a long look this week.

jsirois commented 5 months ago

N.B.: I've noticed a few things and will be pushing a few individually reviewable commits here at the tail. None of these change the nature of the large 1st commit; so any review effort there is not wasted.

jsirois commented 5 months ago

Ok, with 120fe0d I think this is complete and I'll stay hands off until there is time to review.

benjyw commented 5 months ago

Nice! I'll take a close look over the next couple of days.

jsirois commented 5 months ago

I've piece-meal responded. I'll have full time to respond to all comments on 2/26 / 2/27.

Some of the questions, though are answered by tests in this PR if you get impatient.

jsirois commented 5 months ago

I've piece-meal responded. I'll have full time to respond to all comments on 2/26 / 2/27.

@huonw I've fully responded as far as I can tell saving for the case of what to do with --dry-run when there is no lock file yet (the create case). I still have no good idea there besides the current inconsistency. Certainly printing out the contents of the lock that would be written, in JSON form or the sentency form that update --dry-run uses, seems not useful. Perhaps just outputting something like a pip freeze?:

Would lock 37 projects:
  ansicolors 1.1.8
  cowsay 5.0
  foo 1
  bar 1
  ...
  spam 1

... I actually went with this in c1e26df

jsirois commented 5 months ago

Ok, all of @huonw's feedback is addressed AFAICT; so ready for round 2 of feedback.

jsirois commented 5 months ago

Tests are bloody, but it appears to be a GitHub outage with action repos timing out during clone and ghcr.io also timing out when downloading Docker images.

cburroughs commented 5 months ago

Thank you! I'll take a long look this week.

Still looking and prototyping; I hope to have more useful feedback early next week.

benjyw commented 5 months ago

Sorry for the delay, got back from a ski trip yesterday, so doing a deep dive on this now.

cburroughs commented 5 months ago

Thank you! This is great and I've been kicking the tires so to speak and try it out with our internal lockfile. A few in progress comments so far.

Conflicting output

While syncing an internal lock file I've gotten this apparently conflicting message:

Updates for lock generated by universal:
  There were no updates for mypy

But I've been unable to figure out a smaller reproduction case so far.

Extras/Constraints

A bug where a requirement with extras resulted in an invalid tmp constraints file for the sync.

$ cat tmp/dhub.txt 
ansicolors==1.1.5
acryl-datahub[datahub-rest,feast,postgres,snowflake]==0.12.1.4
altair>=4.2.0,<5.2
$ cat tmp/dhub-c.txt 
awscli >= 1.30.0
boto3 >= 1.33.0
botocore >= 1.33.0
cachetools >= 5.3.0
click >= 8.1.0
coverage >= 7.3.0
dash >= 2.10.0
dash_bootstrap_components >= 1.4.0
flake8-pep585 >= 0.1.0
freezegun >= 1.3.0
graphene >= 3.0
holidays >= 0.30
ipython >= 8.0.0
kubernetes >= 28.0.0
loguru >= 0.6.0
matplotlib >= 3.7.0
pytest-mock >= 3.9.0
pytest-xdist >= 3.0.0
pytz >= 2023.3
requests >= 2.20.0
ruamel.yaml >= 0.17.0
s3fs >= 2023.1.0
setuptools >= 69.0.0
shap >= 0.40.0
xarray >= 2023.10.1
$ python3.10 -m pex.cli lock create '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac --prefer-binary  -r tmp/dhub.txt --constraints tmp/dhub-c.txt   -o tmp/dub.json
$ python3.10 -m pex.cli lock sync '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac --prefer-binary  -r tmp/dhub.txt --constraints tmp/dhub-c.txt  --lock tmp/dub.json
Encountered 1 error updating tmp/dub.json:
1.) cp310-cp310-manylinux_2_37_x86_64: /tmp/lock_update.32vgd7eh.constraints.txt line 1:
acryl-datahub[datahub-rest,feast,postgres,snowflake]==0.12.1.4

Constraint files do not support VCS, URL or local project requirementsand they do not support requirements with extras. Search for 'We are also changing our support for Constraints Files' here: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020.

Interpreter Constraints and friends

I see 120fe0d23552863b34750866db7cc79376d0fffb but I'm not sure how these lock options are intended to be "sync-able" changes. That is which can switch to "new" values while trying to hold the versions constant, or only only values that are subsets of the original range.

Naive example trying to "upgrade" Python versions with --interpreter-constraints. (I'm not sure this should be a supported operation by lock sync, just that I don't know how someone looking at --help would know.) In contrast, --pip-version can be changed on a new sync.

$ cat tmp/a-req.txt 
ansicolors==1.1.5
requests
watchfiles
$ python3.10 -m pex.cli lock create '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac --no-build --wheel -r  tmp/a-req.txt -o tmp/a.json
$ cat tmp/a-req.txt 
ansicolors==1.1.8
requests
watchfiles
$ python3.10 -m pex.cli lock sync '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.11.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac -r   tmp/a-req.txt --lock tmp/a.json 
Failed to resolve compatible artifacts from lock tmp/a.json for 1 target:
1. /usr/bin/python3.11:
    Failed to resolve all requirements for cp311-cp311-manylinux_2_37_x86_64 interpreter at /usr/bin/python3.11 from tmp/a.json:

Configured with:
    build: False
    use_wheel: True

Dependency on watchfiles not satisfied, 1 incompatible candidate found:
1.) watchfiles 0.21 (via: watchfiles) does not have any compatible artifacts:
    https://files.pythonhosted.org/packages/70/76/8d124e14cf51af4d6bba926c7473f253c6efd1539ba62577f079a2d71537/watchfiles-0.21.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
    https://files.pythonhosted.org/packages/14/d0/662800e778ca20e7664dd5df57751aa79ef18b6abb92224b03c8c2e852a6/watchfiles-0.21.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
    https://files.pythonhosted.org/packages/18/c4/ad5ad16cad900a29aaa792e0ed121ff70d76f74062b051661090d88c6dfd/watchfiles-0.21.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
    https://files.pythonhosted.org/packages/37/17/4825999346f15d650f4c69093efa64fb040fbff4f706a20e8c4745f64070/watchfiles-0.21.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    https://files.pythonhosted.org/packages/41/0e/3333b986b1889bb71f0e44b3fac0591824a679619b8b8ddd70ff8858edc4/watchfiles-0.21.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    https://files.pythonhosted.org/packages/4e/d2/769254ff04ba88ceb179a6e892606ac4da17338eb010e85ca7a9c3339234/watchfiles-0.21.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
    https://files.pythonhosted.org/packages/5b/79/ecd0dfb04443a1900cd3952d7ea6493bf655c2db9a0d3736a5d98a15da39/watchfiles-0.21.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
    https://files.pythonhosted.org/packages/62/66/7463ceb43eabc6deaa795c7969ff4d4fd938de54e655035483dfd1e97c84/watchfiles-0.21.0-pp310-pypy310_pp73-macosx_10_7_x86_64.whl
    https://files.pythonhosted.org/packages/6e/85/ea2a035b7d86bf0a29ee1c32bc2df8ad4da77e6602806e679d9735ff28cb/watchfiles-0.21.0-cp310-cp310-macosx_10_7_x86_64.whl
    https://files.pythonhosted.org/packages/92/ff/75cc1b30c5abcad13a2a72e75625ec619c7a393028a111d7d24dba578d5e/watchfiles-0.21.0-cp310-cp310-musllinux_1_1_aarch64.whl
    https://files.pythonhosted.org/packages/9a/65/12cbeb363bf220482a559c48107edfd87f09248f55e1ac315a36c2098a0f/watchfiles-0.21.0-cp310-cp310-musllinux_1_1_x86_64.whl
    https://files.pythonhosted.org/packages/b5/e5/240e5eb3ff0ee3da3b028ac5be2019c407bdd0f3fdb02bd75fdf3bd10aff/watchfiles-0.21.0-cp310-cp310-macosx_11_0_arm64.whl
    https://files.pythonhosted.org/packages/f7/4b/b90dcdc3bbaf3bb2db733e1beea2d01566b601c15fcf8e71dfcc8686c097/watchfiles-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
    https://files.pythonhosted.org/packages/fe/a3/42686af3a089f34aba35c39abac852869661938dae7025c1a0580dfe0fbf/watchfiles-0.21.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl
jsirois commented 5 months ago

@cburroughs for Extras/Constraints you did not provide the contents of --constraints tmp/dhub-c.txt. Can you please provide everything needed to repro? Aha - you did. My eye skipped over the break.

jsirois commented 4 months ago

An update since its been a while again:

I've not looked at "Conflicting output" and "Extras/Constraints" since those both look small / easy. I have been working on "Interpreter Constraints and friends" which is hard. It's not hard for Pants' use case of --style universal locks or even --style {strict,sources} locks with 1 locked resolve in the lock file - I have those working with tests. Pex supports multiple --style {strict,sources} locked resolves in 1 lock file though, and that case is taking time to even fully understand. I had a full day today to poke at it, but I'm AFK through the 13th. On that next 14th-17th stint I'll cut my losses on the morning of the 15th if I haven't sussed that case and ship with a failure in sync for that case as the starting point and add the functionality to cover the case in a subsequent release.

jsirois commented 4 months ago

Ok, I am going to proceed without support for lockfiles with multiple locked resolves, that case did prove too much to get working in a way I'm satisfied with yesterday.

jsirois commented 4 months ago

@cburroughs this is good to go afaict.

jsirois commented 4 months ago

Ok, I looked more deeply at "Extras/Constraints" and although I cannot repro using that case, I do see a bug in the code by inspection. I'll add a failing test and fix.

jsirois commented 4 months ago

Alright @cburroughs this should be truly good to hammer now. The only outstanding known issue is the cosmetic "updates: no updates" message.

cburroughs commented 4 months ago

Alright @cburroughs this should be truly good to hammer now. The only outstanding known issue is the cosmetic "updates: no updates" message.

Thanks! Have been swamped but will try to get my hammer out tomorrow.

cburroughs commented 4 months ago

Conflicting output

Still haven't figured out what triggers this

Extras/Constraints

:+1:

Sorry for my slowness. My pattern here has been to hammer on a large internal lockfile, find something that errors out, and then tease out if that's a real thing that happens in public. Usually that has involved a bunch of stumbling over internal packages, local PyPi registries, PEBCAK, or other things that don't generalize. I think I've found a case where if the transitive dependencies are strict (==) it interferes with the generated constraints. That is in this example I think the user intent is reasonably clear ("change the version of this single package") but the sync has a conflict:

$ python3.10 -m pex.cli lock create '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac  'acryl-datahub[postgres,snowflake]==0.12.1.5' -o tmp6/lock.json
$ python3.10 -m pex.cli lock sync '--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython==3.10.*' --style=universal --pip-version=23.3.2 --resolver-version pip-2020-resolver --target-system=linux --target-system=mac  'acryl-datahub[postgres,snowflake]==0.12.1.4' --lock tmp6/lock.json
ERROR: Given the lock requirements:
acryl-datahub[postgres,snowflake]==0.12.1.4

The following lock update constraints could not all be satisfied:
acryl-datahub==0.12.1.4
acryl-datahub-classify==0.0.9
acryl-sqlglot==20.4.1.dev14
aiohttp==3.9.3
aiosignal==1.3.1
altair==4.2
anyio==3.7.1
appnope==0.1.4
argon2-cffi==23.1
argon2-cffi-bindings==21.2
asn1crypto==1.5.1
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2
avro==1.11.3
avro-gen3==0.7.11
beautifulsoup4==4.12.3
bleach==6.1
blis==0.7.11
cached-property==1.5.2
catalogue==2.0.10
certifi==2024.2.2
cffi==1.16
charset-normalizer==3.3.2
click==8.1.7
click-default-group==1.2.4
click-spinner==0.1.10
colorama==0.4.6
comm==0.2.2
confection==0.1.4
cryptography==42.0.5
cymem==2.0.8
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
deprecated==1.2.14
docker==7
entrypoints==0.4
exceptiongroup==1.2
executing==2.0.1
expandvars==0.12
fastjsonschema==2.19.1
filelock==3.13.3
frozenlist==1.4.1
geoalchemy2==0.14.6
great-expectations==0.15.50
greenlet==3.0.3
humanfriendly==10
idna==3.6
ijson==3.2.3
importlib-metadata==7.1
ipaddress==1.0.23
ipykernel==6.17.1
ipython==8.21
ipython-genutils==0.2
ipywidgets==8.1.2
iso3166==2.1.1
jedi==0.19.1
jinja2==3.1.3
jsonpatch==1.33
jsonpointer==2.4
jsonref==1.1
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter-client==7.4.9
jupyter-core==4.12
jupyter-server==1.24
jupyterlab-pygments==0.3
jupyterlab-widgets==3.0.10
langcodes==3.3
makefun==1.15.2
markupsafe==2.1.5
marshmallow==3.21.1
matplotlib-inline==0.1.6
mistune==3.0.2
mixpanel==4.10.1
msal==1.28
multidict==6.0.5
murmurhash==1.0.10
mypy-extensions==1
nbclassic==1
nbclient==0.6.3
nbconvert==7.16.3
nbformat==5.10.2
nest-asyncio==1.6
notebook==6.5.6
notebook-shim==0.2.4
numpy==1.26.4
packaging==24
pandas==2.2.1
pandocfilters==1.5.1
parso==0.8.3
pathlib-abc==0.1.1
pathy==0.11
pexpect==4.9
phonenumbers==8.13
platformdirs==3.11
preshed==3.0.9
progressbar2==4.4.2
prometheus-client==0.20
prompt-toolkit==3.0.43
psutil==5.9.8
psycopg2-binary==2.9.9
ptyprocess==0.7
pure-eval==0.2.2
py==1.11
pycountry==23.12.11
pycparser==2.21
pydantic==1.10.14
pygments==2.17.2
pyjwt==2.8
pyopenssl==24.1
pyparsing==3.1.2
python-dateutil==2.9.post0
python-stdnum==1.20
python-utils==3.8.2
pytz==2024.1
pyyaml==6.0.1
pyzmq==24.0.1
referencing==0.34
requests==2.31
requests-file==2
rpds-py==0.18
ruamel-yaml==0.17.17
schwifty==2024.1.1.post0
scipy==1.12
send2trash==1.8.2
sentry-sdk==1.43
setuptools==69.2
six==1.16
smart-open==6.4
sniffio==1.3.1
snowflake-connector-python==3.7.1
snowflake-sqlalchemy==1.5.1
sortedcontainers==2.4
soupsieve==2.5
spacy==3.5
spacy-legacy==3.0.12
spacy-loggers==1.0.5
sqlalchemy==1.4.52
sqlparse==0.4.4
srsly==2.4.8
stack-data==0.6.3
tabulate==0.9
termcolor==2.4
terminado==0.18.1
thinc==8.1.12
tinycss2==1.2.1
toml==0.10.2
tomlkit==0.12.4
toolz==0.12.1
tornado==6.4
tqdm==4.66.2
traitlets==5.2.1.post0
typer==0.7
typing-extensions==4.10
typing-inspect==0.9
tzdata==2024.1
tzlocal==5.2
urllib3==1.26.18
vininfo==1.8
wasabi==1.1.2
wcwidth==0.2.13
webencodings==0.5.1
websocket-client==1.7
widgetsnbextension==4.0.10
wrapt==1.16
yarl==1.9.4
zipp==3.18.1

Encountered 1 error updating tmp6/lock.json:
1.) cp310-cp310-manylinux_2_37_x86_64: pid 17028 -> /home/ecsb/.pex/venvs/5c1db138e499727676fa1d0ca449202de8f6f34d/5985ed09b49a653d6596b0e14d134c5456cf1a9f/bin/python -sE /home/ecsb/.pex/venvs/5c1db138e499727676fa1d0ca449202de8f6f34d/5985ed09b49a653d6596b0e14d134c5456cf1a9f/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /home/ecsb/.pex/pip/23.3.2/pip_cache --log /tmp/pex-pip-log.6yqis34i/pip.log download --dest /tmp/tmp_a25uxoz/usr.bin.python3.10 --constraint /tmp/lock_update.53w8bkah.constraints.txt acryl-datahub[postgres,snowflake]==0.12.1.4 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Cannot install acryl-datahub[postgres,snowflake]==0.12.1.4 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

 The conflict is caused by:
     acryl-datahub[postgres,snowflake] 0.12.1.4 depends on spacy==3.4.3; extra == "snowflake"
     The user requested (constraint) spacy==3.5

 To fix this you could try to:
 1. loosen the range of package versions you've specified
 2. remove package versions to allow pip attempt to solve the dependency conflict
jsirois commented 4 months ago

@cburroughs my next work stint runs the afternoon of 3/27 through 4/1. I'll take a look then.

jsirois commented 4 months ago

Okay, @cburroughs that last behavior you mentioned is what you get. You can fix by following the reccomendation (and then once more when you get a new but similar conflict) to arrive at a successful:

:; python3.10 -m pex.cli lock sync --indent 2 --interpreter-constraint "CPython==3.10.*" --style universal --pip-version 23.3.2 --resolver-version pip-2020-resolver --target-system linux --target-system mac "acryl-datahub[postgres,snowflake]==0.12.1.4" spacy==3.4.3 "wasabi>=0.9.1,<1.1.0" --lock lock.json
Updates for lock generated by universal:
  Updated acryl-datahub from 0.12.1.5 to 0.12.1.4
  Updated spacy from 3.5 to 3.4.3
  Updated wasabi from 1.1.2 to 0.10.1
Updates to lock input requirements:
  Updated 'acryl-datahub[postgres,snowflake]==0.12.1.5' to 'acryl-datahub[postgres,snowflake]==0.12.1.4'
  Added 'spacy==3.4.3'
  Added 'wasabi<1.1.0,>=0.9.1'

The issue here is that acryl-datahub pins a whole heck of alot of deps (instead of, say, constraining to a non-breaking semver range):

:; unzip -qc ~/downloads/acryl_datahub-0.12.1.4-py3-none-any.whl acryl_datahub-0.12.1.4.dist-info/METADATA | grep Requires | grep -E "==[0-9]" | cut -d';' -f1 | sort -u
Requires-Dist: acryl-datahub-airflow-plugin ==0.12.1.4
Requires-Dist: acryl-datahub-classify ==0.0.9
Requires-Dist: acryl-pyhive[hive-pure-sasl] ==0.6.16
Requires-Dist: acryl-sqlglot ==20.4.1.dev14
Requires-Dist: avro-gen3 ==0.7.11
Requires-Dist: avro-gen3 ==0.7.11
Requires-Dist: black ==22.12.0
Requires-Dist: boto3-stubs[glue,s3,sagemaker,sts] ==1.28.15
Requires-Dist: elasticsearch ==7.13.4
Requires-Dist: flake8-bugbear ==23.3.12
Requires-Dist: google-cloud-datacatalog-lineage ==0.2.2
Requires-Dist: lark[regex] ==1.1.4
Requires-Dist: looker-sdk ==23.0.0
Requires-Dist: msal ==1.22.0
Requires-Dist: mypy ==1.0.0
Requires-Dist: mypy-boto3-sagemaker ==1.28.15
Requires-Dist: spacy ==3.4.3
Requires-Dist: sql-metadata ==2.2.2
Requires-Dist: sqllineage ==1.3.8
Requires-Dist: sqlparse ==0.4.4
Requires-Dist: types-click ==0.1.12
Requires-Dist: vertica-sqlalchemy-dialect[vertica-python] ==0.0.8.1

That sort of mass pinning will lead to issues when the pins change in another version given that Pex updates have these rules:

  1. Pex is not a resolver, it uses Pip to do resolving; as such Pex cannot backtrack, etc. Pip does all that opaquely.
  2. The pex3 lock {update,sync} commands attempt to do a minimal update by masking off unchanged portions of the resolve (as can be divined from the input requirements alone anyhow) with constraints pinning.

Afaict, to have these sorts of situations resolve automatically would either require Pex tries, fails, retries switching constraints pins to semver range pins? But that would lead to more disruptive lock updates growing akin to re-running pex3 lock create - which I think is pointedly the thing you're trying to avoid / fix.

So, how do you wish to proceed? Afaict Pex cannot do better than this without becoming a resolver itself which is not in the cards in Pex 2.x. The user has to be able to read the Pip conflict output and do what I did above or else Pants needs to have an escape hatch to re-run pex3 lock create for locks that are overly constrained preventing update. Maybe though there is some clever strategy that uses constraints ranging instead of pinning to cut down on lock disturbance but still allow updates.

jsirois commented 4 months ago

@cburroughs I may move forward with this iteration this afternoon. I'm getting uncomfortable with the number of changes queued behind a release and it seems like Pants is at liberty to skip this initial release of pex3 lock sync if you find improvements or bugs that we can follow up with.

cburroughs commented 4 months ago

(Yeah some of these large packages with narrow pins are challenging to deal with in a few different ways.)

The mental model of "Pex is not a resolver and thus can't 'backtrack'" makes sense to me. I'll have to think about how to present that through Pants in a way that's helpful without getting in the users way, but that was the essential design problem there anyway.

Thanks for putting this all together.

jsirois commented 4 months ago

I'll have to think about how to present that through Pants in a way that's helpful without getting in the users way

One thing to point out is rm lock.json && pex3 lock sync ... reverts to the old Pants behavior of re-locking every time; so, at worst, if a user can't figure out Pip conflict messages and Pants gives some guidance, you can always work your way out of an overly constrained lock by re-creating it this way even if pex3 lock sync ... is the only command Pants uses behind the scenes.