pex-tool / pex

A tool for generating .pex (Python EXecutable) files, lock files and venvs.
https://docs.pex-tool.org/
Apache License 2.0
2.53k stars 258 forks source link

Don't unconditionally prepend `""` to a venv PEX's `sys.path` #1983

Closed thejcannon closed 1 year ago

thejcannon commented 1 year ago

https://github.com/pantsbuild/pex/pull/1832#discussion_r1025363133 captures the location of the bug.

sys.path behavior is documented https://docs.python.org/3/library/sys.html#sys.path which shows the prepending to sys.path is dependent on how python is invoked.

And to further prove the point:

$ cat foo.py
print(__import__("sys").path)

$ python -m venv venv
...

$ venv/bin/python foo.py
['/home/josh/work/pants', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']

$ venv/bin/python -m foo
['/home/josh/work/pants', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']

$ MYCODE=$(cat foo.py) &&  venv/bin/python -c "$MYCODE"
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']
benjyw commented 1 year ago

Related: https://github.com/pantsbuild/pants/issues/17531

jsirois commented 1 year ago

@thejcannon, your example shell session is useful and is also true all the way back to Python 2.7 - almost:

$ cat foo.py
import sys; print("\n".join(sys.path))
$ python2.7 foo.py
/home/jsirois/dev/pantsbuild/jsirois-pex
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
$ python2.7 -m foo

/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
$ python2.7 -c "$(cat foo.py)"

/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages

In other words, all three of those modes insert CWD at the head of sys.path with sometimes CWD spelled as an absolute path and sometimes as ""; so when PEX acts as an interpreter (via either forcing with PEX_INTERPRETER=1 or else when it has no entry point defined), it should do the same. Currently zipapp mode does this correctly just here: https://github.com/pantsbuild/pex/blob/67fc7e05315fc347bcf269712814f160c6035efe/pex/pex.py#L627-L631

But venv mode does it unconditionally as you highlight here: https://github.com/pantsbuild/pex/blob/67fc7e05315fc347bcf269712814f160c6035efe/pex/venv/pex.py#L443-L444