pantsbuild / pants

The Pants Build System
https://www.pantsbuild.org
Apache License 2.0
3.32k stars 636 forks source link

generate-lockfiles crash when using :all: value in resolves_to_no_binary #20323

Open lilyhahnotg opened 10 months ago

lilyhahnotg commented 10 months ago

Describe the bug

I seem to be getting some issues with the python.resolves_to_no_binary option when generating lockfiles for my python-default resolve. Setting the value for a specific package works:

[python.resolves_to_no_binary]
python-default = ["pyspark"]

But setting all packages to resolves_to_no_binary (attempting to replicate the no_binary option on previous versions) causes an error:

pants --keep-sandboxes=on_failure generate-lockfiles
Engine traceback:
  in `generate-lockfiles` goal

ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:

stderr:
pid 28647 -> /Users/lilyhahn/.cache/pants/named_caches/pex_root/venvs/ef49b90fd05eeaf0726e2b94dc62642722fe5d73/43788ff4c4e0c9f7527522d7b727648118d65dc5/bin/python -sE /Users/lilyhahn/.cache/pants/named_caches/pex_root/venvs/ef49b90fd05eeaf0726e2b94dc62642722fe5d73/43788ff4c4e0c9f7527522d7b727648118d65dc5/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/lilyhahn/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/30/r2sc86257xgfnvdl1h3d_p280000gp/T/pants-sandbox-6FNXkB/.tmp/pex-pip-log.vzfo35ez/pip.log download --dest /private/var/folders/30/r2sc86257xgfnvdl1h3d_p280000gp/T/pants-sandbox-6FNXkB/.tmp/tmpizd_4rut/Users.lilyhahn.miniconda3.envs.***.bin.python3.7 --requirement __pip_args.txt JSON-log-formatter==0.5.1 beautifulsoup4==4.12.2 boto3 pandas==1.1.5 paramiko==3.1.0 py4j==0.10.9.3 pyspark==3.2.1 pytest requests==2.26.0 selenium==4.2.0 types-paramiko types-pytz types-requests==2.26.0 --index-url https://pypi.org/simple/ --retries 5 --timeout 15 exited with 1 and STDERR:
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

The error from the subprocess seems to be

ERROR: Could not open requirements file: [Errno 2] No such file or directory: '__pip_args.txt'

Pants version Which version of Pants are you using? 2.18.0

OS Are you encountering the bug on MacOS, Linux, or both? MacOS

Additional info Add any other information about the problem here, such as attachments or links to gists, if relevant.

benjyw commented 10 months ago

I finally got around to looking at this, sorry for the delay. Unfortunately I can't reproduce it. The context is that __pip_args.txt is a file containing args that Pants passes to pip. But that error is from a subprocess that pip spawned. I can't think of a circumstance in which a pip subprocess (presumably invoked to build a wheel from an sdist) would know about that file.

So we'll need you to create a small public github repo that demonstrates the problem. This seems to be related to specific circumstances you're encountering and I don't know how to reproduce.