Issues in Docker environment after failure

[Peder2911]

Describe the bug I am getting random failures when using a Docker environment for building Pexes and running tests. The failures sometimes only happen on one try, and don't appear on subsequent retries. Sometimes the failures are more persistent, and don't go away until I've deleted the cache and pants-related dotfiles in the repo.

The errors being thrown are:

stderr:
SyntaxError: Non-UTF-8 code starting with '\xc3' in file /pants-sandbox/pants-sandb
ox-LhQ4ne/./pex on line 3, but no encoding declared; see https://python.org/dev/pep
s/pep-0263/ for details

stderr:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 190, in _run_module_as_main
    mod_name, mod_spec, code = _get_main_module_details(_Error)
  File "/usr/local/lib/python3.9/runpy.py", line 239, in _get_main_module_details
    return _get_module_details(main_name)
  File "/usr/local/lib/python3.9/runpy.py", line 130, in _get_module_details
    spec = importlib.util.find_spec(mod_name)
  File "/usr/local/lib/python3.9/importlib/util.py", line 103, in find_spec
    return _find_spec(fullname, parent_path)
  File "<frozen importlib._bootstrap>", line 925, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1423, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1397, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1378, in _legacy_get_spec
  File "<frozen importlib._bootstrap>", line 423, in spec_from_loader
  File "<frozen importlib._bootstrap_external>", line 716, in spec_from_file_location
  File "<frozen zipimport>", line 191, in get_filename
  File "<frozen zipimport>", line 709, in _get_module_code
  File "<frozen zipimport>", line 536, in _get_data
FileNotFoundError: [Errno 2] No such file or directory: '/pants-sandbox/pants-sandbox-1wF3q0/./pex'

stderr:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/pants-sandbox/pants-sandbox-vzFC1B/./pex/__main__.py", line 89, in <module>
ModuleNotFoundError: No module named 'pex'

Pants version I am using Pants 2.16.0.dev7

OS I am on an M1 Mac

Additional info I am using Docker Desktop 20.10.23 All of my Docker builds are being run on the linux/amd64 platform.

My pants.toml looks like this: https://gist.github.com/Peder2911/8250bb84257ed6163e0d7efb09155322

[Peder2911]

This happens both when I run ./pants test and ./pants package

[Peder2911]

New error message:

stderr:
Failed to load an installed wheel layout from /pants-sandbox/pants-sandbox-5HpirV/p
ytest.pex/.deps/pytest_cov-3.0.0-py3-none-any.whl/.layout.json: [Errno 20] Not a di
rectory: '/pants-sandbox/pants-sandbox-5HpirV/pytest.pex/.deps/pytest_cov-3.0.0-py3
-none-any.whl/.layout.json'

[Peder2911]

My build environment is declared like this: https://gist.github.com/Peder2911/1c9ea87dd330a645235defc98895be13

[Peder2911]

Aha, maybe not so random. Issues start whenever something fails during preparation for the test / packaging. Seems to leave the build container in a bad state. Removing the container and deleting .p* from the repository stops the issues.

[benjyw]

Thanks for this report, and for the further debugging. Do you have examples of prep failures that trigger this?

[stuhood]

This is a dupe of #18162: sorry for the trouble! That issue is fixed in 2.15.0, but not yet on main/2.16.x.