pex3 venv create inconsistently creates a plain venv

[zmanji]

This inconsistency is a minor issue but surprised me recently.

Creating a venv via a requirement produces a plain venv

pex3 venv create  --force -d ./tenv --no-build 'mypy'
ls ./tenv 
bin        include    lib        pyvenv.cfg

Creating a pex of the requirement and then creating a venv from the pex repository produces a pex venv

pex --pip-version 23.2 --resolver-version pip-2020-resolver --no-build 'mypy' -o test.pex
pex3 venv create --force -d ./tenv  --no-build --pex-repository ./test.pex 'mypy'
ls ./tenv
PEX-INFO    __main__.py bin         include     lib         pex         pyvenv.cfg

I would expect both steps to be consistent but maybe I missed something in the documentation

[jsirois]

Yeah, so for a PEX repository, install scope comes into play and it defaults to all (code and deps). You were expecting a default of deps / consistency: https://github.com/pantsbuild/pex/blob/4eb5c9aa25c6a695bf55263ab239189b720cebaf/pex/venv/installer_options.py#L19-L37

Can you elaborate on your surprise? Did this cause a bug?

[zmanji]

I ended up observing two different Docker image hashes for two steps that should have been equivalent. It took me a bit to realize that the produced venvs were different because of the pex entrypoint.

If this is intentional I don't think this is a bug at all just a bit unexpected because I didn't see it in the help output/docs.

[jsirois]

To close the loop here, using --scope deps does net you the same venv:

$ pex3 venv create  --force -d ./tenv --no-build 'mypy'
$ pex --pip-version 23.2 --resolver-version pip-2020-resolver --no-build 'mypy' -o test.pex
$ pex3 venv create --force -d ./tenv2 --no-build --pex-repository ./test.pex 'mypy' --scope deps

$ diff -r -u ./tenv ./tenv2
diff -r -u ./tenv/bin/activate ./tenv2/bin/activate
--- ./tenv/bin/activate 2023-12-26 21:23:52.399479551 -0800
+++ ./tenv2/bin/activate        2023-12-26 21:24:46.169471677 -0800
@@ -39,10 +39,10 @@
 if [ "$OSTYPE" = "cygwin" ] || [ "$OSTYPE" = "msys" ] ; then
     # transform D:\path\to\venv to /d/path/to/venv on MSYS
     # and to /cygdrive/d/path/to/venv on Cygwin
-    export VIRTUAL_ENV=$(cygpath "/home/jsirois/tenv")
+    export VIRTUAL_ENV=$(cygpath "/home/jsirois/tenv2")
 else
     # use the path as-is
-    export VIRTUAL_ENV="/home/jsirois/tenv"
+    export VIRTUAL_ENV="/home/jsirois/tenv2"
 fi

 _OLD_VIRTUAL_PATH="$PATH"
@@ -59,9 +59,9 @@

 if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
     _OLD_VIRTUAL_PS1="${PS1:-}"
-    PS1="(tenv) ${PS1:-}"
+    PS1="(tenv2) ${PS1:-}"
     export PS1
-    VIRTUAL_ENV_PROMPT="(tenv) "
+    VIRTUAL_ENV_PROMPT="(tenv2) "
     export VIRTUAL_ENV_PROMPT
 fi

diff -r -u ./tenv/bin/activate.csh ./tenv2/bin/activate.csh
--- ./tenv/bin/activate.csh     2023-12-26 21:23:52.399479551 -0800
+++ ./tenv2/bin/activate.csh    2023-12-26 21:24:46.169471677 -0800
@@ -9,7 +9,7 @@
 # Unset irrelevant variables.
 deactivate nondestructive

-setenv VIRTUAL_ENV "/home/jsirois/tenv"
+setenv VIRTUAL_ENV "/home/jsirois/tenv2"

 set _OLD_VIRTUAL_PATH="$PATH"
 setenv PATH "$VIRTUAL_ENV/bin:$PATH"
@@ -18,8 +18,8 @@
 set _OLD_VIRTUAL_PROMPT="$prompt"

 if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then
-    set prompt = "(tenv) $prompt"
-    setenv VIRTUAL_ENV_PROMPT "(tenv) "
+    set prompt = "(tenv2) $prompt"
+    setenv VIRTUAL_ENV_PROMPT "(tenv2) "
 endif

 alias pydoc python -m pydoc
diff -r -u ./tenv/bin/activate.fish ./tenv2/bin/activate.fish
--- ./tenv/bin/activate.fish    2023-12-26 21:23:52.399479551 -0800
+++ ./tenv2/bin/activate.fish   2023-12-26 21:24:46.169471677 -0800
@@ -33,7 +33,7 @@
 # Unset irrelevant variables.
 deactivate nondestructive

-set -gx VIRTUAL_ENV "/home/jsirois/tenv"
+set -gx VIRTUAL_ENV "/home/jsirois/tenv2"

 set -gx _OLD_VIRTUAL_PATH $PATH
 set -gx PATH "$VIRTUAL_ENV/bin" $PATH
@@ -56,7 +56,7 @@
         set -l old_status $status

         # Output the venv prompt; color taken from the blue of the Python logo.
-        printf "%s%s%s" (set_color 4B8BBE) "(tenv) " (set_color normal)
+        printf "%s%s%s" (set_color 4B8BBE) "(tenv2) " (set_color normal)

         # Restore the return status of the previous command.
         echo "exit $old_status" | .
@@ -65,5 +65,5 @@
     end

     set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV"
-    set -gx VIRTUAL_ENV_PROMPT "(tenv) "
+    set -gx VIRTUAL_ENV_PROMPT "(tenv2) "
 end
diff -r -u ./tenv/bin/dmypy ./tenv2/bin/dmypy
--- ./tenv/bin/dmypy    2023-12-26 21:24:08.589476423 -0800
+++ ./tenv2/bin/dmypy   2023-12-26 21:24:46.379471762 -0800
@@ -1,4 +1,4 @@
-#!/home/jsirois/tenv/bin/python3.12 -sE
+#!/home/jsirois/tenv2/bin/python3.12 -sE
 # -*- coding: utf-8 -*-
 import importlib
 import sys
diff -r -u ./tenv/bin/mypy ./tenv2/bin/mypy
--- ./tenv/bin/mypy     2023-12-26 21:24:08.589476423 -0800
+++ ./tenv2/bin/mypy    2023-12-26 21:24:46.379471762 -0800
@@ -1,4 +1,4 @@
-#!/home/jsirois/tenv/bin/python3.12 -sE
+#!/home/jsirois/tenv2/bin/python3.12 -sE
 # -*- coding: utf-8 -*-
 import importlib
 import sys
diff -r -u ./tenv/bin/mypyc ./tenv2/bin/mypyc
--- ./tenv/bin/mypyc    2023-12-26 21:24:08.589476423 -0800
+++ ./tenv2/bin/mypyc   2023-12-26 21:24:46.379471762 -0800
@@ -1,4 +1,4 @@
-#!/home/jsirois/tenv/bin/python3.12 -sE
+#!/home/jsirois/tenv2/bin/python3.12 -sE
 # -*- coding: utf-8 -*-
 import importlib
 import sys
diff -r -u ./tenv/bin/stubgen ./tenv2/bin/stubgen
--- ./tenv/bin/stubgen  2023-12-26 21:24:08.589476423 -0800
+++ ./tenv2/bin/stubgen 2023-12-26 21:24:46.379471762 -0800
@@ -1,4 +1,4 @@
-#!/home/jsirois/tenv/bin/python3.12 -sE
+#!/home/jsirois/tenv2/bin/python3.12 -sE
 # -*- coding: utf-8 -*-
 import importlib
 import sys
diff -r -u ./tenv/bin/stubtest ./tenv2/bin/stubtest
--- ./tenv/bin/stubtest 2023-12-26 21:24:08.589476423 -0800
+++ ./tenv2/bin/stubtest        2023-12-26 21:24:46.379471762 -0800
@@ -1,4 +1,4 @@
-#!/home/jsirois/tenv/bin/python3.12 -sE
+#!/home/jsirois/tenv2/bin/python3.12 -sE
 # -*- coding: utf-8 -*-
 import importlib
 import sys
diff -r -u ./tenv/pyvenv.cfg ./tenv2/pyvenv.cfg
--- ./tenv/pyvenv.cfg   2023-12-26 21:23:52.399479551 -0800
+++ ./tenv2/pyvenv.cfg  2023-12-26 21:24:46.169471677 -0800
@@ -2,4 +2,4 @@
 include-system-site-packages = false
 version = 3.12.1
 executable = /usr/bin/python3.12
-command = /usr/bin/python3.12 -m venv --without-pip /home/jsirois/tenv
+command = /usr/bin/python3.12 -m venv --without-pip /home/jsirois/tenv2
$

I do agree that from a resolver standpoint, you'd expect Pip, lock file and PEX repository to work the same, as they do in other contexts. Here the consistency is between how pex-tools test.pex venv ... works and pex3 venv create --pex-repository test.pex ... works. I agree it probably would have been better to default --scope to deps for pex3 venv create, but I blew that and the ship has sailed.