Pex external venv should copy wheel files

[zmanji]

Peeled out of https://github.com/pantsbuild/pex/issues/2312#issuecomment-1871722718

Thinking about it an external venv should copy files from the wheel cache instead of hardlinks / symlinks.

In my case I use pex to create venvs for tools like VSCode. I could choose to edit a file in a venv for debugging purposes and if I do, I inadvertently corrupt the wheel cache.

Example:

PEX_SCRIPT='pex3' pex venv create -vv --copies --force -d ./tenv  --no-build 'cowsay==6.1'

Then remove the first line (import re) of

./tenv/lib/python3.10/site-packages/cowsay/main.py

Then the venv is broken when I run:

./tenv/bin/cowsay -t 'test'
Traceback (most recent call last):
  File "/home/zmanji/tmp/./tenv/bin/cowsay", line 14, in <module>
    sys.exit(entry_point())
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/__main__.py", line 28, in cli
    char_funcs[args.character](args.text)
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/__init__.py", line 24, in func
    draw(str(text), char_lines)
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/main.py", line 54, in draw
    if len(re.sub(r'\s', '', text)) == 0:
NameError: name 're' is not defined

Even if I delete the venv and recreate it it still suffers from the same problem

rm -rf ./tenv 
PEX_SCRIPT='pex3' pex venv create -vv --copies --force -d ./tenv  --no-build 'cowsay==6.1'
./tenv/bin/cowsay -t 'test'  
Traceback (most recent call last):
  File "/home/zmanji/tmp/./tenv/bin/cowsay", line 14, in <module>
    sys.exit(entry_point())
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/__main__.py", line 28, in cli
    char_funcs[args.character](args.text)
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/__init__.py", line 24, in func
    draw(str(text), char_lines)
  File "/home/zmanji/tmp/tenv/lib/python3.10/site-packages/cowsay/main.py", line 54, in draw
    if len(re.sub(r'\s', '', text)) == 0:
NameError: name 're' is not defined

The only way to fix this is to delete the pex wheel cache.

A way to fix this is to overload the existing --copies flag to also copy files from the wheel cache or to introduce another flag to signal this and change this function https://github.com/pantsbuild/pex/blob/4eb5c9aa25c6a695bf55263ab239189b720cebaf/pex/pep_376.py#L350-L404

to do copies instead of links.

[jsirois]

I'm going to use --site-packages-copies to stay ~consistent with pex --venv-site-packages-copies ... used for venvs internal to the PEX_ROOT, the difference being those try to hard link 1st and fall back to copy, whereas these external venvs will just copy.

[zmanji]

Once this is fixed, pex will be a pretty ideal tool for managing dev venvs.

[jsirois]

Ok, the --site-packages-copies option is now available in the Pex 2.12.0 release.