Virtualenv does not work correctly on python 2.7 from python.org and new macOS 12 Monterey

s-vitaliy commented 2 years ago

Issue

Hello, we faced an issuie with virtualenv when it is used in python, downloaded form python.org: https://www.python.org/download/releases/2.7/ (Mac Installer disk image)

If virtualenv is created by virtualenv, installed in this package, pip ignores virtualenv and always installs packages into system site-packages: testbot@mac12-1005 ~ % ./broken_venv/bin/python -m pip install pytest \<cut output> testbot@mac12-1005 ~ % ./broken_venv/bin/python -m pip show pytest \<cut output> Location: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

If virtuelenv is created by built-in python (/usr/bin/python), virtualenv works correctly:

testbot@mac12-1005 ~ % ./correct_venv/bin/python -m pip install pytest \<cut output> testbot@mac12-1005 ~ % ./correct_venv/bin/python -m pip show pytest \<cut output> Location: /Users/testbot/correct_venv/lib/python2.7/site-packages

For some reason, new site-packages in broken_venv does not contain site-packages of new virtualenv:

testbot@mac12-1005 ~ % ./broken_venv/bin/python -m site
sys.path = [
    '/Users/testbot',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
    '/Users/testbot/Library/Python/2.7/lib/python/site-packages',
    '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages',
]
USER_BASE: '/Users/testbot/Library/Python/2.7' (exists)
USER_SITE: '/Users/testbot/Library/Python/2.7/lib/python/site-packages' (exists)
ENABLE_USER_SITE: True
testbot@mac12-1005 ~ % ./correct_venv/bin/python -m site
sys.path = [
    '/Users/testbot',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
    '/Users/testbot/correct_venv/lib/python2.7/site-packages',
    '/Library/Python/2.7/site-packages',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC',
]
USER_BASE: '/Users/testbot/Library/Python/2.7' (exists)
USER_SITE: '/Users/testbot/Library/Python/2.7/lib/python/site-packages' (exists)
ENABLE_USER_SITE: False

Also I noticed that broken virtualenv does not detect base_prefix: "base_prefix": null and correct virtualenv contains "base_prefix": "/System/Library/Frameworks/Python.framework/Versions/2.7" (see detailed of bouth commands below)

We use test framework that need to be configured to access mac os UI for UI testing automation, so we cannot use directly /usr/bin/python and need to install pyton via dmg image.

Is it possible to fix this issue or create workaround to make virtualenv work correctly in our case?

Environment

OS:

testbot@mac12-1005 ~ % sw_vers
ProductName:    macOS
ProductVersion: 12.0.1
BuildVersion:   21A559

pip list of the host python where virtualenv is installed:

Package                                Version
-------------------------------------- -----------
argh                                   0.26.2
attrs                                  21.2.0
Automat                                20.2.0
backports.functools-lru-cache          1.6.4
beautifulsoup4                         4.9.3
cffi                                   1.15.0
configparser                           4.0.2
constantly                             15.1.0
contextlib2                            0.6.0.post1
cryptography                           3.3.2
distlib                                0.3.4
EasyProcess                            0.3
enum34                                 1.1.6
fastjsonrpc                            0.3.0
filelock                               3.2.1
futures                                3.3.0
google                                 2.0.2
grpcio                                 1.24.1
hyperlink                              21.0.0
idna                                   2.10
importlib-metadata                     2.1.2
importlib-resources                    3.3.1
incremental                            17.5.0
ipaddress                              1.0.23
jsonstruct                             0.2a1
lxml                                   4.1.1
monotonic                              1.5
numpy                                  1.14.1
pathlib2                               2.3.6
pathtools                              0.1.2
pika                                   1.1.0
Pillow                                 5.1.0
pip                                    20.3.4
platformdirs                           2.0.2
protobuf                               3.13.0
psutil                                 5.4.3
pyasn1                                 0.4.8
pyasn1-modules                         0.2.8
pyatomac                               2.0.7
pycparser                              2.21
PyHamcrest                             1.10.1
pymongo                                3.12.1
pyobjc                                 4.1
pyobjc-core                            5.3
pyobjc-framework-Accounts              4.1
pyobjc-framework-AddressBook           4.1
pyobjc-framework-AppleScriptKit        4.1
pyobjc-framework-AppleScriptObjC       4.1
pyobjc-framework-ApplicationServices   4.1
pyobjc-framework-Automator             4.1
pyobjc-framework-AVFoundation          4.1
pyobjc-framework-AVKit                 4.1
pyobjc-framework-CalendarStore         4.1
pyobjc-framework-CFNetwork             4.1
pyobjc-framework-CloudKit              4.1
pyobjc-framework-Cocoa                 5.3
pyobjc-framework-Collaboration         4.1
pyobjc-framework-ColorSync             4.1
pyobjc-framework-Contacts              4.1
pyobjc-framework-ContactsUI            4.1
pyobjc-framework-CoreBluetooth         4.1
pyobjc-framework-CoreData              4.1
pyobjc-framework-CoreLocation          4.1
pyobjc-framework-CoreML                4.1
pyobjc-framework-CoreServices          4.1
pyobjc-framework-CoreSpotlight         4.1
pyobjc-framework-CoreText              4.1
pyobjc-framework-CoreWLAN              4.1
pyobjc-framework-CryptoTokenKit        4.1
pyobjc-framework-DictionaryServices    4.1
pyobjc-framework-DiskArbitration       4.1
pyobjc-framework-EventKit              4.1
pyobjc-framework-ExceptionHandling     4.1
pyobjc-framework-ExternalAccessory     4.1
pyobjc-framework-FinderSync            4.1
pyobjc-framework-FSEvents              4.1
pyobjc-framework-GameCenter            4.1
pyobjc-framework-GameController        4.1
pyobjc-framework-GameKit               4.1
pyobjc-framework-GameplayKit           4.1
pyobjc-framework-ImageCaptureCore      4.1
pyobjc-framework-IMServicePlugIn       4.1
pyobjc-framework-InputMethodKit        4.1
pyobjc-framework-InstallerPlugins      4.1
pyobjc-framework-InstantMessage        4.1
pyobjc-framework-Intents               4.1
pyobjc-framework-IOSurface             4.1
pyobjc-framework-iTunesLibrary         4.1
pyobjc-framework-LatentSemanticMapping 4.1
pyobjc-framework-LaunchServices        4.1
pyobjc-framework-libdispatch           4.1
pyobjc-framework-LocalAuthentication   4.1
pyobjc-framework-MapKit                4.1
pyobjc-framework-MediaAccessibility    4.1
pyobjc-framework-MediaLibrary          4.1
pyobjc-framework-MediaPlayer           4.1
pyobjc-framework-ModelIO               4.1
pyobjc-framework-MultipeerConnectivity 4.1
pyobjc-framework-NetFS                 4.1
pyobjc-framework-NetworkExtension      4.1
pyobjc-framework-NotificationCenter    4.1
pyobjc-framework-OpenDirectory         4.1
pyobjc-framework-Photos                4.1
pyobjc-framework-PhotosUI              4.1
pyobjc-framework-PreferencePanes       4.1
pyobjc-framework-PubSub                4.1
pyobjc-framework-QTKit                 4.1
pyobjc-framework-Quartz                4.1
pyobjc-framework-SafariServices        4.1
pyobjc-framework-SceneKit              4.1
pyobjc-framework-ScreenSaver           4.1
pyobjc-framework-ScriptingBridge       4.1
pyobjc-framework-SearchKit             4.1
pyobjc-framework-Security              4.1
pyobjc-framework-SecurityFoundation    4.1
pyobjc-framework-SecurityInterface     4.1
pyobjc-framework-ServiceManagement     4.1
pyobjc-framework-Social                4.1
pyobjc-framework-SpriteKit             4.1
pyobjc-framework-StoreKit              4.1
pyobjc-framework-SyncServices          4.1
pyobjc-framework-SystemConfiguration   4.1
pyobjc-framework-Vision                4.1
pyobjc-framework-WebKit                4.1
pyOpenSSL                              17.5.0
pyscreenshot                           0.4.2
python-dateutil                        2.8.1
python-ntlm                            1.1.0
PyYAML                                 5.4.1
reimport                               1.4.1
scandir                                1.10.0
selenium                               3.14.1
service-identity                       17.0.0
setuptools                             43.0.0
setuptools-git                         1.2
singledispatch                         3.7.0
six                                    1.16.0
soupsieve                              1.9.6
subprocess32                           3.5.4
Twisted                                17.9.0
txmongo                                18.0.0
typing                                 3.7.4.1
urllib3                                1.26.7
virtualenv                             20.13.0
watchdog                               0.8.3
zipp                                   1.2.0
zope.interface                         5.4.0

Output of the virtual environment creation

Incorrect virutualenv creation log Make sure to run the creation with -vvv --with-traceback:

testbot@mac12-1005 ~ % /Library/Frameworks/Python.framework/Versions/2.7/bin/python -m virtualenv broken_venv -vvv --with-traceback
112 setup logging to NOTSET [DEBUG report:39]
113 created app data folder /Users/testbot/Library/Application Support/virtualenv [DEBUG __init__:41]
143 find interpreter for spec PythonSpec(path=/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python) [INFO builtin:62]
144 proposed PythonInfo({'base_exec_prefix': None, 'system_stdlib': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'has_venv': False, 'prefix': u'/Library/Frameworks/Python.framework/Versions/2.7', 'stdout_encoding': u'US-ASCII', 'executable': '/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'implementation': u'CPython', 'exec_prefix': u'/Library/Frameworks/Python.framework/Versions/2.7', 'platform': u'darwin', 'version': u'2.7.18 (v2.7.18:8d21aa21f2, Apr 19 2020, 20:48:48) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]', 'sysconfig_paths': {u'platstdlib': u'{platbase}/lib/python{py_version_short}', u'platlib': u'{platbase}/lib/python{py_version_short}/site-packages', u'purelib': u'{base}/lib/python{py_version_short}/site-packages', u'stdlib': u'{base}/lib/python{py_version_short}', u'scripts': u'{base}/bin', u'include': u'{base}/include/python{py_version_short}', u'data': u'{base}'}, 'sysconfig_scheme': None, 'base_prefix': None, 'system_stdlib_platform': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'file_system_encoding': u'utf-8', 'version_info': VersionInfo(major=2, minor=7, micro=18, releaselevel=u'final', serial=0), 'sysconfig_vars': {u'base': u'/Library/Frameworks/Python.framework/Versions/2.7', u'platbase': u'/Library/Frameworks/Python.framework/Versions/2.7', u'PYTHONFRAMEWORK': u'Python', u'py_version_short': u'2.7'}, 'path': [u'', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages'], 'max_size': 9223372036854775807, 'sysconfig': {u'makefile_filename': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile'}, 'real_prefix': None, 'distutils_install': {u'purelib': u'lib/python2.7/site-packages', u'headers': u'include/python2.7/UNKNOWN', u'platlib': u'lib/python2.7/site-packages', u'data': u'', u'scripts': u'bin'}, 'architecture': 64, 'original_executable': u'/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'os': u'posix', 'system_executable': u'/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python'}) [INFO builtin:69]
144 accepted PythonInfo({'base_exec_prefix': None, 'system_stdlib': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'has_venv': False, 'prefix': u'/Library/Frameworks/Python.framework/Versions/2.7', 'stdout_encoding': u'US-ASCII', 'executable': '/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'implementation': u'CPython', 'exec_prefix': u'/Library/Frameworks/Python.framework/Versions/2.7', 'platform': u'darwin', 'version': u'2.7.18 (v2.7.18:8d21aa21f2, Apr 19 2020, 20:48:48) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]', 'sysconfig_paths': {u'platstdlib': u'{platbase}/lib/python{py_version_short}', u'platlib': u'{platbase}/lib/python{py_version_short}/site-packages', u'purelib': u'{base}/lib/python{py_version_short}/site-packages', u'stdlib': u'{base}/lib/python{py_version_short}', u'scripts': u'{base}/bin', u'include': u'{base}/include/python{py_version_short}', u'data': u'{base}'}, 'sysconfig_scheme': None, 'base_prefix': None, 'system_stdlib_platform': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'file_system_encoding': u'utf-8', 'version_info': VersionInfo(major=2, minor=7, micro=18, releaselevel=u'final', serial=0), 'sysconfig_vars': {u'base': u'/Library/Frameworks/Python.framework/Versions/2.7', u'platbase': u'/Library/Frameworks/Python.framework/Versions/2.7', u'PYTHONFRAMEWORK': u'Python', u'py_version_short': u'2.7'}, 'path': [u'', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages'], 'max_size': 9223372036854775807, 'sysconfig': {u'makefile_filename': u'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile'}, 'real_prefix': None, 'distutils_install': {u'purelib': u'lib/python2.7/site-packages', u'headers': u'include/python2.7/UNKNOWN', u'platlib': u'lib/python2.7/site-packages', u'data': u'', u'scripts': u'bin'}, 'architecture': 64, 'original_executable': u'/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'os': u'posix', 'system_executable': u'/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python'}) [DEBUG builtin:71]
147 filesystem is not case-sensitive [DEBUG info:29]
194 create virtual environment via CPython2macOsFramework(dest=/Users/testbot/broken_venv, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
194 create folder /Users/testbot/broken_venv/bin [DEBUG _sync:24]
195 create folder /Users/testbot/broken_venv/include [DEBUG _sync:24]
195 create folder /Users/testbot/broken_venv/lib/python2.7/site-packages [DEBUG _sync:24]
195 write /Users/testbot/broken_venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
195     home = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
195     implementation = CPython [DEBUG pyenv_cfg:38]
195     version_info = 2.7.18.final.0 [DEBUG pyenv_cfg:38]
195     virtualenv = 20.13.0 [DEBUG pyenv_cfg:38]
195     include-system-site-packages = false [DEBUG pyenv_cfg:38]
195     base-prefix = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
195     base-exec-prefix = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
195     base-executable = /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python [DEBUG pyenv_cfg:38]
196 copy /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python to /Users/testbot/broken_venv/bin/python [DEBUG _sync:51]
197 symlink /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py to /Users/testbot/broken_venv/lib/python2.7/os.py [DEBUG _sync:43]
197 symlink /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc to /Users/testbot/broken_venv/lib/python2.7/os.pyc [DEBUG _sync:43]
197 symlink directory /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 to /Users/testbot/broken_venv/include/python2.7 [DEBUG _sync:43]
197 symlink /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile to /Users/testbot/broken_venv/lib/python2.7/config/Makefile [DEBUG _sync:43]
198 symlink /Library/Frameworks/Python.framework/Versions/2.7/Python to /Users/testbot/broken_venv/.Python [DEBUG _sync:43]
198 symlink directory /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload to /Users/testbot/broken_venv/lib/python2.7/lib-dynload [DEBUG _sync:43]
198 create virtualenv import hook file /Users/testbot/broken_venv/lib/python2.7/site-packages/_virtualenv.pth [DEBUG api:95]
199 create /Users/testbot/broken_venv/lib/python2.7/site-packages/_virtualenv.py [DEBUG api:98]
200 change Mach-O for /Users/testbot/broken_venv/bin/python from /Library/Frameworks/Python.framework/Versions/2.7/Python to @executable_path/../.Python [DEBUG mac_os:201]
200 ============================== target debug ============================== [DEBUG session:54]
201 debug via /Users/testbot/broken_venv/bin/python /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv/create/debug.py [DEBUG creator:224]
200 {
  "makefile_filename": "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile",
  "site": "<module 'site' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.pyc'>",
  "datetime": "<module 'datetime' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/datetime.so'>",
  "sys": {
    "executable": "/Users/testbot/broken_venv/bin/python",
    "real_prefix": null,
    "base_exec_prefix": null,
    "exec_prefix": "/Library/Frameworks/Python.framework/Versions/2.7",
    "io_encoding": null,
    "meta_path": [],
    "fs_encoding": "utf-8",
    "prefix": "/Library/Frameworks/Python.framework/Versions/2.7",
    "_base_executable": null,
    "path": [
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages"
    ],
    "base_prefix": null
  },
  "json": "<module 'json' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.pyc'>",
  "version": "2.7.18 (v2.7.18:8d21aa21f2, Apr 19 2020, 20:48:48) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]",
  "os": "<module 'os' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>",
  "math": "<module 'math' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/math.so'>"
} [DEBUG session:55]
486 add seed packages via FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/testbot/Library/Application Support/virtualenv) [INFO session:59]
491 wrote embed update of distribution pip at /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/pip.json [DEBUG via_disk_folder:161]
491 wrote embed update of distribution wheel at /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/wheel.json [DEBUG via_disk_folder:161]
494 wrote embed update of distribution setuptools at /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/setuptools.json [DEBUG via_disk_folder:161]
508 triggered periodic upgrade of pip==20.3.4 (for python 2.7) via background process having PID 1068 [INFO periodic_update:238]
512 got embed update of distribution pip from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/pip.json [DEBUG via_disk_folder:135]
514 triggered periodic upgrade of setuptools==44.1.1 (for python 2.7) via background process having PID 1070 [INFO periodic_update:238]
517 got embed update of distribution setuptools from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/setuptools.json [DEBUG via_disk_folder:135]
2909 triggered periodic upgrade of wheel==0.37.1 (for python 2.7) via background process having PID 1069 [INFO periodic_update:238]
2910 got embed update of distribution wheel from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/wheel.json [DEBUG via_disk_folder:135]
2911 install pip from wheel /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv/seed/wheels/embed/pip-20.3.4-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
2911 Attempting to acquire lock 140627458626064 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:144]
2911 Lock 140627458626064 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:148]
2911 build install image for pip-20.3.4-py2.py3-none-any.whl to /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any [DEBUG base:49]
2923 install wheel from wheel /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv/seed/wheels/embed/wheel-0.37.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
2923 Attempting to acquire lock 140627458603344 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:144]
2923 Lock 140627458603344 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:148]
2923 build install image for wheel-0.37.1-py2.py3-none-any.whl to /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any [DEBUG base:49]
2958 Attempting to release lock 140627458603344 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:178]
2958 Lock 140627458603344 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:181]
2959 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel-0.37.1.virtualenv [DEBUG _sync:51]
2959 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel [DEBUG _sync:51]
2960 install setuptools from wheel /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv/seed/wheels/embed/setuptools-44.1.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
2976 Attempting to acquire lock 140627458601360 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:144]
2976 Lock 140627458601360 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:148]
2976 build install image for setuptools-44.1.1-py2.py3-none-any.whl to /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any [DEBUG base:49]
3031 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel-0.37.1.dist-info [DEBUG _sync:51]
3037 generated console scripts wheel2.7 wheel-2.7 wheel wheel2 [DEBUG base:45]
3086 Attempting to release lock 140627458601360 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:178]
3086 Lock 140627458601360 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:181]
3086 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools-44.1.1.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools-44.1.1.dist-info [DEBUG _sync:51]
3091 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/easy_install.py to /Users/testbot/broken_venv/lib/python2.7/site-packages/easy_install.py [DEBUG _sync:51]
3092 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools [DEBUG _sync:51]
3134 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/pkg_resources to /Users/testbot/broken_venv/lib/python2.7/site-packages/pkg_resources [DEBUG _sync:51]
3149 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools-44.1.1.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools-44.1.1.virtualenv [DEBUG _sync:51]
3172 generated console scripts easy_install-2.7 easy_install2.7 easy_install easy_install2 [DEBUG base:45]
3173 Attempting to release lock 140627458626064 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:178]
3173 Lock 140627458626064 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:181]
3173 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip-20.3.4.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip-20.3.4.virtualenv [DEBUG _sync:51]
3174 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip [DEBUG _sync:51]
3327 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip-20.3.4.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip-20.3.4.dist-info [DEBUG _sync:51]
3332 generated console scripts pip2.7 pip-2.7 pip pip2 [DEBUG base:45]
3333 add activators for Nushell, Python, Fish, CShell, PowerShell, Bash [INFO session:66]
3337 write /Users/testbot/broken_venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
3337    home = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
3337    implementation = CPython [DEBUG pyenv_cfg:38]
3337    version_info = 2.7.18.final.0 [DEBUG pyenv_cfg:38]
3337    virtualenv = 20.13.0 [DEBUG pyenv_cfg:38]
3337    include-system-site-packages = false [DEBUG pyenv_cfg:38]
3337    base-prefix = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
3337    base-exec-prefix = /Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
3337    base-executable = /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python [DEBUG pyenv_cfg:38]
3338 created virtual environment CPython2.7.18.final.0-64 in 3226ms
  creator CPython2macOsFramework(dest=/Users/testbot/broken_venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/testbot/Library/Application Support/virtualenv)
    added seed packages: pip==20.3.4, setuptools==44.1.1, wheel==0.37.1
  activators NushellActivator,PythonActivator,FishActivator,CShellActivator,PowerShellActivator,BashActivator [WARNING __main__:19]

Correct virtualenv creation log

testbot@mac12-1005 ~ % /usr/bin/python -m virtualenv broken_venv -vvv --with-traceback
309 setup logging to NOTSET [DEBUG report:39]
330 find interpreter for spec PythonSpec(path=/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python) [INFO builtin:62]
330 proposed PythonInfo({'base_exec_prefix': None, 'system_stdlib': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'has_venv': False, 'prefix': u'/System/Library/Frameworks/Python.framework/Versions/2.7', 'stdout_encoding': u'US-ASCII', 'executable': '/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'implementation': u'CPython', 'exec_prefix': u'/System/Library/Frameworks/Python.framework/Versions/2.7', 'platform': u'darwin', 'version': u'2.7.18 (default, Oct  2 2021, 04:20:39) \n[GCC Apple LLVM 13.0.0 (clang-1300.0.29.1) [+internal-os, ptrauth-isa=deploymen', 'sysconfig_paths': {u'platstdlib': u'{platbase}/lib/python{py_version_short}', u'platlib': u'{platbase}/lib/python{py_version_short}/site-packages', u'purelib': u'{base}/lib/python{py_version_short}/site-packages', u'stdlib': u'{base}/lib/python{py_version_short}', u'scripts': u'{base}/bin', u'include': u'{base}/include/python{py_version_short}', u'data': u'{base}'}, 'sysconfig_scheme': None, 'base_prefix': None, 'system_stdlib_platform': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'file_system_encoding': u'utf-8', 'version_info': VersionInfo(major=2, minor=7, micro=18, releaselevel=u'final', serial=0), 'sysconfig_vars': {u'base': u'/System/Library/Frameworks/Python.framework/Versions/2.7', u'platbase': u'/System/Library/Frameworks/Python.framework/Versions/2.7', u'PYTHONFRAMEWORK': u'Python', u'py_version_short': u'2.7'}, 'path': [u'', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', u'/Users/testbot/Library/Python/2.7/lib/python/site-packages', u'/Library/Python/2.7/site-packages', u'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', u'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC'], 'max_size': 9223372036854775807, 'sysconfig': {u'makefile_filename': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile'}, 'real_prefix': None, 'distutils_install': {u'purelib': u'lib/python2.7/site-packages', u'headers': u'include/python2.7/UNKNOWN', u'platlib': u'lib/python2.7/site-packages', u'data': u'', u'scripts': u'bin'}, 'architecture': 64, 'original_executable': u'/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'os': u'posix', 'system_executable': u'/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python'}) [INFO builtin:69]
330 accepted PythonInfo({'base_exec_prefix': None, 'system_stdlib': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'has_venv': False, 'prefix': u'/System/Library/Frameworks/Python.framework/Versions/2.7', 'stdout_encoding': u'US-ASCII', 'executable': '/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'implementation': u'CPython', 'exec_prefix': u'/System/Library/Frameworks/Python.framework/Versions/2.7', 'platform': u'darwin', 'version': u'2.7.18 (default, Oct  2 2021, 04:20:39) \n[GCC Apple LLVM 13.0.0 (clang-1300.0.29.1) [+internal-os, ptrauth-isa=deploymen', 'sysconfig_paths': {u'platstdlib': u'{platbase}/lib/python{py_version_short}', u'platlib': u'{platbase}/lib/python{py_version_short}/site-packages', u'purelib': u'{base}/lib/python{py_version_short}/site-packages', u'stdlib': u'{base}/lib/python{py_version_short}', u'scripts': u'{base}/bin', u'include': u'{base}/include/python{py_version_short}', u'data': u'{base}'}, 'sysconfig_scheme': None, 'base_prefix': None, 'system_stdlib_platform': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', 'file_system_encoding': u'utf-8', 'version_info': VersionInfo(major=2, minor=7, micro=18, releaselevel=u'final', serial=0), 'sysconfig_vars': {u'base': u'/System/Library/Frameworks/Python.framework/Versions/2.7', u'platbase': u'/System/Library/Frameworks/Python.framework/Versions/2.7', u'PYTHONFRAMEWORK': u'Python', u'py_version_short': u'2.7'}, 'path': [u'', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', u'/Users/testbot/Library/Python/2.7/lib/python/site-packages', u'/Library/Python/2.7/site-packages', u'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', u'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC'], 'max_size': 9223372036854775807, 'sysconfig': {u'makefile_filename': u'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile'}, 'real_prefix': None, 'distutils_install': {u'purelib': u'lib/python2.7/site-packages', u'headers': u'include/python2.7/UNKNOWN', u'platlib': u'lib/python2.7/site-packages', u'data': u'', u'scripts': u'bin'}, 'architecture': 64, 'original_executable': u'/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', 'os': u'posix', 'system_executable': u'/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python'}) [DEBUG builtin:71]
333 filesystem is not case-sensitive [DEBUG info:29]
388 create virtual environment via CPython2macOsFramework(dest=/Users/testbot/broken_venv, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
389 write /Users/testbot/broken_venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
389     home = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
389     implementation = CPython [DEBUG pyenv_cfg:38]
389     version_info = 2.7.18.final.0 [DEBUG pyenv_cfg:38]
389     virtualenv = 20.13.0 [DEBUG pyenv_cfg:38]
389     include-system-site-packages = false [DEBUG pyenv_cfg:38]
389     base-prefix = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
389     base-exec-prefix = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
389     base-executable = /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python [DEBUG pyenv_cfg:38]
390 remove file /Users/testbot/broken_venv/bin/python [DEBUG _sync:37]
390 copy /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python to /Users/testbot/broken_venv/bin/python [DEBUG _sync:51]
392 remove file /Users/testbot/broken_venv/lib/python2.7/os.py [DEBUG _sync:37]
392 symlink /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py to /Users/testbot/broken_venv/lib/python2.7/os.py [DEBUG _sync:43]
392 remove file /Users/testbot/broken_venv/lib/python2.7/os.pyc [DEBUG _sync:37]
392 symlink /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc to /Users/testbot/broken_venv/lib/python2.7/os.pyc [DEBUG _sync:43]
393 remove file /Users/testbot/broken_venv/include/python2.7 [DEBUG _sync:37]
393 symlink directory /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 to /Users/testbot/broken_venv/include/python2.7 [DEBUG _sync:43]
393 remove file /Users/testbot/broken_venv/lib/python2.7/config/Makefile [DEBUG _sync:37]
393 symlink /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile to /Users/testbot/broken_venv/lib/python2.7/config/Makefile [DEBUG _sync:43]
394 remove file /Users/testbot/broken_venv/.Python [DEBUG _sync:37]
394 symlink /System/Library/Frameworks/Python.framework/Versions/2.7/Python to /Users/testbot/broken_venv/.Python [DEBUG _sync:43]
394 remove file /Users/testbot/broken_venv/lib/python2.7/lib-dynload [DEBUG _sync:37]
394 symlink directory /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload to /Users/testbot/broken_venv/lib/python2.7/lib-dynload [DEBUG _sync:43]
394 create virtualenv import hook file /Users/testbot/broken_venv/lib/python2.7/site-packages/_virtualenv.pth [DEBUG api:95]
395 create /Users/testbot/broken_venv/lib/python2.7/site-packages/_virtualenv.py [DEBUG api:98]
396 change Mach-O for /Users/testbot/broken_venv/bin/python from /System/Library/Frameworks/Python.framework/Versions/2.7/Python to @executable_path/../.Python [DEBUG mac_os:201]
397 ============================== target debug ============================== [DEBUG session:54]
398 debug via /Users/testbot/broken_venv/bin/python /Users/testbot/Library/Python/2.7/lib/python/site-packages/virtualenv/create/debug.py [DEBUG creator:224]
397 {
  "makefile_filename": "/Users/testbot/broken_venv/lib/python2.7/config/Makefile",
  "site": "<module 'site' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.pyc'>",
  "datetime": "<module 'datetime' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/datetime.so'>",
  "sys": {
    "executable": "/Users/testbot/broken_venv/bin/python",
    "real_prefix": "/System/Library/Frameworks/Python.framework/Versions/2.7",
    "base_exec_prefix": "/System/Library/Frameworks/Python.framework/Versions/2.7",
    "exec_prefix": "/Users/testbot/broken_venv",
    "io_encoding": null,
    "meta_path": [
      "<class '_virtualenv._VirtualenvImporter'>"
    ],
    "fs_encoding": "utf-8",
    "prefix": "/Users/testbot/broken_venv",
    "_base_executable": null,
    "path": [
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old",
      "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload",
      "/Users/testbot/broken_venv/lib/python2.7/site-packages"
    ],
    "base_prefix": "/System/Library/Frameworks/Python.framework/Versions/2.7"
  },
  "json": "<module 'json' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.pyc'>",
  "version": "2.7.18 (default, Oct  2 2021, 04:20:39) \n[GCC Apple LLVM 13.0.0 (clang-1300.0.29.1) [+internal-os, ptrauth-isa=deploymen",
  "os": "<module 'os' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>",
  "math": "<module 'math' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/math.so'>"
} [DEBUG session:55]
491 add seed packages via FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/testbot/Library/Application Support/virtualenv) [INFO session:59]
496 got embed update of distribution wheel from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/wheel.json [DEBUG via_disk_folder:135]
497 got embed update of distribution pip from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/pip.json [DEBUG via_disk_folder:135]
497 got embed update of distribution setuptools from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/setuptools.json [DEBUG via_disk_folder:135]
501 got embed update of distribution setuptools from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/setuptools.json [DEBUG via_disk_folder:135]
502 got embed update of distribution wheel from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/wheel.json [DEBUG via_disk_folder:135]
502 got embed update of distribution pip from /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/embed/3/pip.json [DEBUG via_disk_folder:135]
503 install wheel from wheel /Users/testbot/Library/Python/2.7/lib/python/site-packages/virtualenv/seed/wheels/embed/wheel-0.37.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
503 install pip from wheel /Users/testbot/Library/Python/2.7/lib/python/site-packages/virtualenv/seed/wheels/embed/pip-20.3.4-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
504 install setuptools from wheel /Users/testbot/Library/Python/2.7/lib/python/site-packages/virtualenv/seed/wheels/embed/setuptools-44.1.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
505 Attempting to acquire lock 4389624080 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:144]
505 Attempting to acquire lock 4389623568 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:144]
505 Lock 4389624080 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:148]
505 Attempting to acquire lock 4389622032 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:144]
505 Lock 4389623568 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:148]
506 Attempting to release lock 4389624080 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:178]
506 Attempting to release lock 4389623568 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:178]
506 Lock 4389623568 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any.lock [DEBUG _api:181]
506 Lock 4389622032 acquired on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:148]
506 Lock 4389624080 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:181]
507 Attempting to release lock 4389622032 on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:178]
507 Lock 4389622032 released on /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any.lock [DEBUG _api:181]
507 uninstall existing distribution pip-20.3.4 from /Users/testbot/broken_venv/lib/python2.7/site-packages [DEBUG base:161]
507 uninstall existing distribution setuptools-44.1.1 from /Users/testbot/broken_venv/lib/python2.7/site-packages [DEBUG base:161]
508 uninstall existing distribution wheel-0.37.1 from /Users/testbot/broken_venv/lib/python2.7/site-packages [DEBUG base:161]
565 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel-0.37.1.virtualenv [DEBUG _sync:51]
566 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel [DEBUG _sync:51]
578 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/wheel-0.37.1.dist-info [DEBUG _sync:51]
583 generated console scripts wheel2.7 wheel-2.7 wheel wheel2 [DEBUG base:45]
585 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools-44.1.1.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools-44.1.1.dist-info [DEBUG _sync:51]
589 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/easy_install.py to /Users/testbot/broken_venv/lib/python2.7/site-packages/easy_install.py [DEBUG _sync:51]
590 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools [DEBUG _sync:51]
623 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip-20.3.4.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip-20.3.4.virtualenv [DEBUG _sync:51]
624 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip [DEBUG _sync:51]
629 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/pkg_resources to /Users/testbot/broken_venv/lib/python2.7/site-packages/pkg_resources [DEBUG _sync:51]
639 copy /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/setuptools-44.1.1-py2.py3-none-any/setuptools-44.1.1.virtualenv to /Users/testbot/broken_venv/lib/python2.7/site-packages/setuptools-44.1.1.virtualenv [DEBUG _sync:51]
642 generated console scripts easy_install-2.7 easy_install2.7 easy_install easy_install2 [DEBUG base:45]
765 copy directory /Users/testbot/Library/Application Support/virtualenv/wheel/2.7/image/1/CopyPipInstall/pip-20.3.4-py2.py3-none-any/pip-20.3.4.dist-info to /Users/testbot/broken_venv/lib/python2.7/site-packages/pip-20.3.4.dist-info [DEBUG _sync:51]
770 generated console scripts pip2.7 pip-2.7 pip pip2 [DEBUG base:45]
770 add activators for Nushell, Python, Fish, CShell, PowerShell, Bash [INFO session:66]
774 write /Users/testbot/broken_venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
774     home = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
774     implementation = CPython [DEBUG pyenv_cfg:38]
774     version_info = 2.7.18.final.0 [DEBUG pyenv_cfg:38]
774     virtualenv = 20.13.0 [DEBUG pyenv_cfg:38]
774     include-system-site-packages = false [DEBUG pyenv_cfg:38]
774     base-prefix = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
774     base-exec-prefix = /System/Library/Frameworks/Python.framework/Versions/2.7 [DEBUG pyenv_cfg:38]
774     base-executable = /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python [DEBUG pyenv_cfg:38]
774 created virtual environment CPython2.7.18.final.0-64 in 466ms
  creator CPython2macOsFramework(dest=/Users/testbot/broken_venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/testbot/Library/Application Support/virtualenv)
    added seed packages: pip==20.3.4, setuptools==44.1.1, wheel==0.37.1
  activators NushellActivator,PythonActivator,FishActivator,CShellActivator,PowerShellActivator,BashActivator [WARNING __main__:19]

gaborbernat commented 2 years ago

Is it possible to fix this issue or create workaround to make virtualenv work correctly in our case?

Feel free to put in a PR to fix it. :+1:

s-vitaliy commented 2 years ago

@gaborbernat, can you help me to find ideas how to find the reason, why this happening?

gaborbernat commented 2 years ago

Sorry, I do not have time for that at this point in time.

s-vitaliy commented 2 years ago

@gaborbernat, ok, thank you.

yusufertekin commented 2 years ago

Hello, we are having the same issue. It only happens with python2.7 installed from python.org. Seems like virtualenv is not creating its own copy of python and pip instead using the global one installed under /Library/Frameworks/Python.framework/Versions/2.7/.
Overriding [global] target helps to install packages under virtualenv but then python won't find any packages even we include virtualenv folder in sys path. @gaborbernat If you can guide me with some tips, I'd love to contribute with a PR to solve this issue.

gaborbernat commented 2 years ago

Step one would be to run the test suite against that python and see what breaks 🤔 and then hunt the problems down one by one.

yusufertekin commented 2 years ago

I tried however there are too many errors and going so deep, I think I'm not making any progress. Here is the log for who picks this up.

ERROR tests/unit/create/test_creator.py::test_create_no_seed[old_virtualenv-cpython2-mac-framework-symlinks-isolated] - RuntimeError: failed to query /U...
ERROR tests/unit/create/test_creator.py::test_create_no_seed[old_virtualenv-cpython2-mac-framework-copies-global] - RuntimeError: failed to query /Users...
ERROR tests/unit/create/test_creator.py::test_create_no_seed[old_virtualenv-cpython2-mac-framework-copies-isolated] - RuntimeError: failed to query /Use...
ERROR tests/unit/activation/test_powershell.py::test_powershell[no_prompt] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py2...
ERROR tests/integration/test_zipapp.py::test_zipapp_create[pip] - RuntimeError: could not find a python to build zipapp
ERROR tests/integration/test_zipapp.py::test_zipapp_create[app-data] - RuntimeError: could not find a python to build zipapp
ERROR tests/integration/test_zipapp.py::test_zipapp_help - RuntimeError: could not find a python to build zipapp
FAILED tests/unit/activation/test_csh.py::test_csh[no_prompt] - AssertionError: /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/python
FAILED tests/unit/activation/test_bash.py::test_bash[with_prompt] - AssertionError: /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/python
FAILED tests/unit/activation/test_bash.py::test_bash[no_prompt] - AssertionError: /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/python
FAILED tests/unit/discovery/py_info/test_py_info.py::test_py_info_cached_symlink - assert 4 == (1 + 1)
FAILED tests/unit/activation/test_csh.py::test_csh[with_prompt] - AssertionError: /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/python
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2.7-64-bin] - RuntimeError: failed to detect cpython2.7.18-64|...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2-bin] - RuntimeError: failed to detect cpython2.7.18-64|cpyth...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2.7-bin] - RuntimeError: failed to detect cpython2.7.18-64|cpy...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2.7.18-bin] - RuntimeError: failed to detect cpython2.7.18-64...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2.7.18-64-bin] - RuntimeError: failed to detect cpython2.7.18...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2.7-bin] - RuntimeError: failed to detect cpython2.7.18-64|cp...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2-bin] - RuntimeError: failed to detect cpython2.7.18-64|cpyt...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2-64-bin] - RuntimeError: failed to detect cpython2.7.18-64|c...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2.7.18-64-bin] - RuntimeError: failed to detect cpython2.7.18-...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2.7.18-bin] - RuntimeError: failed to detect cpython2.7.18-64|...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[python-2-64-bin] - RuntimeError: failed to detect cpython2.7.18-64|cp...
FAILED tests/unit/discovery/py_info/test_py_info_exe_based_of.py::test_discover_ok[CPython-2.7-64-bin] - RuntimeError: failed to detect cpython2.7.18-64...
FAILED tests/unit/create/test_creator.py::test_pth_in_site_vs_PYTHONPATH - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/...
FAILED tests/unit/create/test_creator.py::test_get_site_packages - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/pyth...
FAILED tests/unit/create/test_creator.py::test_create_vcs_ignore_override - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27...
FAILED tests/unit/create/test_creator.py::test_os_path_sep_not_allowed - assert 'SystemExit: ' in 'RuntimeError: failed to query /Users/yusufertekin/Sit...
FAILED tests/unit/create/test_creator.py::test_zip_importer_can_import_setuptools - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/....
FAILED tests/unit/create/test_creator.py::test_getsitepackages_system_site - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py2...
FAILED tests/unit/create/test_creator.py::test_create_distutils_cfg[cpython2-mac-framework] - RuntimeError: failed to query /Users/yusufertekin/Sites/vi...
FAILED tests/unit/create/test_creator.py::test_py_pyc_missing[True-False] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27...
FAILED tests/unit/create/test_creator.py::test_no_preimport_threading - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin...
FAILED tests/unit/create/test_creator.py::test_python_path[on] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/python...
FAILED tests/unit/create/test_creator.py::test_create_clear_resets[no_clear-cpython2-mac-framework] - RuntimeError: failed to query /Users/yusufertekin/...
FAILED tests/unit/create/test_creator.py::test_create_no_seed[root-cpython2-mac-framework-symlinks-isolated] - RuntimeError: failed to query /Users/yusu...
FAILED tests/unit/create/test_creator.py::test_py_pyc_missing[False-False] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py2...
FAILED tests/unit/create/test_creator.py::test_debug_bad_virtualenv - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/p...
FAILED tests/unit/create/test_creator.py::test_create_no_seed[root-cpython2-mac-framework-copies-isolated] - RuntimeError: failed to query /Users/yusufe...
FAILED tests/unit/create/test_creator.py::test_create_no_seed[root-cpython2-mac-framework-symlinks-global] - RuntimeError: failed to query /Users/yusufe...
FAILED tests/unit/create/test_creator.py::test_create_vcs_ignore_exists_override - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.t...
FAILED tests/unit/create/test_creator.py::test_create_vcs_ignore_exists - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/b...
FAILED tests/unit/create/test_creator.py::test_destination_not_write_able - assert 'SystemExit: ' in 'RuntimeError: failed to query /Users/yusufertekin/...
FAILED tests/unit/create/test_creator.py::test_python_path[off] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/pytho...
FAILED tests/unit/create/test_creator.py::test_prompt_set[None-cpython2-mac-framework] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtual...
FAILED tests/unit/create/test_creator.py::test_create_clear_resets[clear-cpython2-mac-framework] - RuntimeError: failed to query /Users/yusufertekin/Sit...
FAILED tests/unit/create/test_creator.py::test_create_no_seed[root-cpython2-mac-framework-copies-global] - RuntimeError: failed to query /Users/yusufert...
FAILED tests/unit/create/test_creator.py::test_destination_exists_file - assert 'SystemExit: ' in 'RuntimeError: failed to query /Users/yusufertekin/Sit...
FAILED tests/unit/create/test_creator.py::test_create_long_path - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/pytho...
FAILED tests/unit/create/test_creator.py::test_py_pyc_missing[True-True] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/...
FAILED tests/unit/create/test_creator.py::test_prompt_set[magic-cpython2-mac-framework] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtua...
FAILED tests/unit/create/test_creator.py::test_py_pyc_missing[False-True] - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27...
FAILED tests/unit/discovery/test_discovery.py::test_discovery_fallback_ok - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27...
FAILED tests/unit/discovery/test_discovery.py::test_relative_path - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/pyt...
FAILED tests/unit/discovery/test_discovery.py::test_discovery_via_path[upper] - assert None is not None
FAILED tests/unit/discovery/test_discovery.py::test_discovery_via_path[mixed] - assert None is not None
FAILED tests/unit/discovery/test_discovery.py::test_discovery_via_path[lower] - assert None is not None
FAILED tests/unit/create/via_global_ref/test_build_c_ext.py::test_can_build_c_extensions[cpython2-mac-framework] - RuntimeError: failed to query /Users/...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_download_manual_ignores_pre_release - IOError: [Errno 2] No such file or directory: '/Librar...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_download_stop_with_embed - IOError: [Errno 2] No such file or directory: '/Library/Framework...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_do_update_skip_already_done - IOError: [Errno 2] No such file or directory: '/Library/Framew...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_download_manual_stop_after_one_download - IOError: [Errno 2] No such file or directory: '/Li...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_download_periodic_stop_at_first_usable_with_previous_minor - IOError: [Errno 2] No such file...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_pick_periodic_update - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.t...
FAILED tests/unit/seed/wheels/test_periodic_update.py::test_download_periodic_stop_at_first_usable - IOError: [Errno 2] No such file or directory: '/Lib...
FAILED tests/unit/config/test_ini.py::test_ini_can_be_overwritten_by_flag - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27...
FAILED tests/unit/seed/embed/test_bootstrap_link_via_app_data.py::test_base_bootstrap_link_via_app_data_no[setuptools] - RuntimeError: failed to query /...
FAILED tests/unit/seed/embed/test_bootstrap_link_via_app_data.py::test_seed_link_via_app_data[True] - RuntimeError: failed to query /Users/yusufertekin/...
FAILED tests/unit/seed/embed/test_bootstrap_link_via_app_data.py::test_base_bootstrap_link_via_app_data_no[pip] - RuntimeError: failed to query /Users/y...
FAILED tests/unit/seed/embed/test_bootstrap_link_via_app_data.py::test_seed_link_via_app_data[False] - RuntimeError: failed to query /Users/yusufertekin...
FAILED tests/unit/seed/embed/test_bootstrap_link_via_app_data.py::test_base_bootstrap_link_via_app_data_no[wheel] - RuntimeError: failed to query /Users...
FAILED tests/unit/seed/embed/test_pip_invoke.py::test_base_bootstrap_via_pip_invoke[pip] - IOError: [Errno 2] No such file or directory: '/Library/Frame...
FAILED tests/unit/seed/embed/test_pip_invoke.py::test_base_bootstrap_via_pip_invoke[wheel] - IOError: [Errno 2] No such file or directory: '/Library/Fra...
FAILED tests/unit/seed/embed/test_pip_invoke.py::test_base_bootstrap_via_pip_invoke[] - IOError: [Errno 2] No such file or directory: '/Library/Framewor...
FAILED tests/unit/seed/embed/test_pip_invoke.py::test_base_bootstrap_via_pip_invoke[setuptools] - IOError: [Errno 2] No such file or directory: '/Librar...
FAILED tests/unit/create/test_creator.py::test_cross_major - RuntimeError: failed to query /usr/local/bin/python3 with code 2 err: "/Library/Frameworks/...
FAILED tests/integration/test_run_int.py::test_app_data_pinning - RuntimeError: failed to query /Users/yusufertekin/Sites/virtualenv/.tox/py27/bin/pytho...
========================================= 70 failed, 202 passed, 18 skipped, 2 warnings, 7 error in 62.43 seconds =========================================

gaborbernat commented 2 years ago

For what it is worth it might look like a lot but there's a high degree of duplication in between there. So you might just need to fix two or three to fix it. I recommend picking one test and focus on fixing that, once it's fixed run the suite again and start over 👍

yusufertekin commented 2 years ago

@gaborbernat I have some other deadlines to meet and to be honest, making 0 progress in 4 hours kinda scared me :) However, I suspect two things

something wrong with sys.path configuration, seems like virtualenv path is not getting appended into sys.path and assertions of our path (syspath - systempath) return an empty list.
And magically at some point (after running an external command) tests mess up with installed virtualenv directory and the rest of the tests won't be able to find virtualenv directory. (IOError: [Errno 2] No such file or directory)

isac322 commented 2 years ago

@s-vitaliy I found temporal solution.

pip install -U virtualenv
python2 -m virtualenv venv
lipo -remove arm64e venv/bin/python -o venv/bin/python

lipo manipulates Mach-O universal binary to disable arm64e (Apple Silicon) and venv/bin/python will always run with Rosetta 2.

jasmeen19 commented 2 years ago

@isac322 The given workaround dint work for me

fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: input file (env/bin/python) must be a fat file when the -remove option is specified Do I need to do something here to get this working?

yusufertekin commented 2 years ago

@isac322 Thank you for looking into it. I wanted to let you know that the same problem happens no matter if an M1 or intel-based chip is used. It started to happen specifically after the Monterey upgrade. Also, I'm getting the same error on lipo command.

yusufertekin commented 2 years ago

Hello @gaborbernat, I found the problem but I'm not sure how to solve it. I have a hack fix. I'll share it below. What happens is;

sys.prefix is not set to VIRTUALENV_HOME instead it stays '/Library/Frameworks/Python.framework/Versions/2.7'

On sys documentation for python 2.7, it states that this is set on compile-time. So I believe unless any explicit intervention seems like there is no way to set this because we basically copy python compiled files into the virtualenv via global self do.

We could have updated it using the custom site.py created for python2 however, that's not loaded due to sys.prefix not pointing the virtualenv directory where custom site.py placed.

What I have done is, I created sitecustomizate.py under /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ and added this

import sys
import os
env_path = os.getenv('VIRTUAL_ENV')
if env_path:
    sys.path = filter(
      lambda val: val != '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', 
      sys.path
    )  # Remove all occurrences of '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages'
  sys.prefix = env_path  # Set prefix as virtualenv home
  sys.path.append(env_path + '/lib/python2.7/site-packages')

I don't have time at the moment to continue working on this. There must be a much better way to solve this. If you can guide me, I'd love to create a PR to contribute.

Copying custom site.py content under sitecustomize.py not remapping paths also due to reloading global site.py, global site-packages gets added to the PATH even though it becomes the last.

There are so many tests failing one main reason is what I describe above another is we are using Tox to run tests which uses virtualenv :) to create virtual environment for tests. Due to this bug, tox actually uses the global space to install dependencies and also virtualenv custom build. That's why I was seeing, global virtualenv module magically got broken or removed.

UPDATE: I've found out that some of the dependencies still gets installed under global site-packages, i.e. numpy Also tests not working, tox cannot create package. So this is still not a workaround :(

I'm not sure how Monterey upgrade affected this prefix problem. I found a StackOverflow post stating "If the file lib/python/dyn-load is found in that directory or any of its parent directories, that directory is set to be to be sys.exec_prefix on Linux or Mac. If the file lib/python/os.py is is found in the directory or any of its subdirectories, that directory is set to be sys.prefix on Linux, Mac, and Windows, with sys.exec_prefix set to the same value as sys.prefix on Windows. "

Maybe this has changed, therefore, path manipulation by the site.py not working anymore.

yusufertekin commented 2 years ago

@gaborbernat Sorry to bug you. Adding sitecustomize.py not actually solve the problem because there are many packages directly importing site.py of python source.

Because I haven’t figured out why after Monterey upgrade custom site.py is not loading before global python site.py, I had to update site.py of python source code to work smarter like python3 site.py does and be able to read pyvenv.cfg file if present. I'm attaching it here. site_py.txt

With this change, no need to use fixed path for macos. So had to change _executables method under src/virtualenv/create/via_global_ref/builtin/cpython/mac_os.py

    @classmethod
    def _executables(cls, interpreter):
        for _, targets, must, when in super(CPythonmacOsFramework, cls)._executables(interpreter):
            # Make sure we use the embedded interpreter inside the framework, even if sys.executable points to the
            # stub executable in ${sys.prefix}/bin.
            # See http://groups.google.com/group/python-virtualenv/browse_thread/thread/17cab2f85da75951
            host_exe = Path(interpreter.system_executable)
            major, minor = interpreter.version_info.major, interpreter.version_info.minor
            targets = OrderedDict(
                (i, None) for i in ["python", "python{}".format(major), "python{}.{}".format(major, minor), host_exe.name]
            )
            must = RefMust.COPY if interpreter.version_info.major == 2 else RefMust.NA
            yield host_exe, list(targets.keys()), must, RefWhen.ANY

gaborbernat commented 2 years ago

Most of the magic on Python 2 happens here https://github.com/pypa/virtualenv/blob/main/src/virtualenv/create/via_global_ref/builtin/python2/site.py#L10-L24 :thinking: so the question is why is this file not doing what you want.

yusufertekin commented 2 years ago

@gaborbernat because it's not getting executed. I tried to manually update site.py and mimic this file functionality into python original site.py but even then it didn't work as expected. Two reasons,

sys.prefix was still pointing to the global python path
Even with manual fix on sys.prefix on the load of site.py Somehow global site packages still shows up in the sys.path

gaborbernat commented 2 years ago

sys.prefix should be altered by executing this file I've pointed too. By design of how python binary works on V2 it should get executed. If Apple patched/altered that behaviour then all hope is lost and there's no way to get it working.

yusufertekin commented 2 years ago

I found a StackOverflow post stating "If the file lib/python/dyn-load is found in that directory or any of its parent directories, that directory is set to be to be sys.exec_prefix on Linux or Mac. If the file lib/python/os.py is is found in the directory or any of its subdirectories, that directory is set to be sys.prefix on Linux, Mac, and Windows, with sys.exec_prefix set to the same value as sys.prefix on Windows. "

So I was assuming it'd get executed as you described but it's not. I try with Intel mac, also with an intel mac not upgraded Monterey, it works fine. Only with Monterey, it's not getting executed and sys.prefix points to global Python. Thank you for confirming.

gaborbernat commented 2 years ago

Yeah, you can find more details here https://youtu.be/o1Vue9CWRxU https://gaborbernat.github.io/euro_python_2019/#/

yusufertekin commented 2 years ago

Thank you these are great. I appreciate.

hynek commented 2 years ago

JFTR, as Apple dropped Python 2 from the base installation few days ago with the 12.3 update, the old workaround (that I used too because I was too lazy to investigate) just disappeared and there's no viable way to run Python 2 on macOS.

The problem is not specific to python.org, I've tested it against Homebrew and asdf and they all show the same behavior.

I wonder if this is the sign of the universe to drop Python 2 from all my FOSS projects. 🤪

gaborbernat commented 2 years ago

I wonder if this is the sign of the universe to drop Python 2 from all my FOSS projects. 🤪

Seems like it. virtualenv goal here was to support Python for 18 months after its EOL. And that seems to pass so anything that works now is chance driven and community supported only. The next release of virtualenv will likely drop support for Python 2.

juanb001 commented 2 years ago

Hopefully someone figures out how to get virtualenv properly working on macOS 12.3.

Previously, my virtual environments were working flawlessly on 12.2.1 (intel Mac). After 12.3 removed the native python and I re-installed from python.org, I could not get virtualenvs to work as they did on 12.2.1. As the original poster mentions, python installs new software into the global python2 environment, instead of only in the virtualenv.

nickhutchinson commented 2 years ago

It looks like virtualenv's trick of adding a symlink to Python.framework/Versions/2.7/Python in the virtualenv root and rewriting the dependent dylib list of my/venv/bin/python* with the path of this symlink no longer works in 12.3.

I suspect macOS 12.3 affected the behaviour of the following block in getpath.c: https://github.com/python/cpython/blob/v2.7.18/Modules/getpath.c#L445 where sys.prefix is derived, and as a result Python ends up skipping over the virtual environment. In macOS < 12.3, NSLibraryNameForModule() probably returned the path to the /path/to/my/venv/.Python symlink that virtualenv creates in the in the virtual environment root, and so that's what Python based sys.prefix on. But in 12.3, NSLibraryNameForModule() returns the resolved path, e.g., /Library/Frameworks/Python.framework/Versions/2.7/Python. (I don't have a 12.2 system to test against so this is mostly guesswork.)

As a workaround:

$ cp /Library/Frameworks/Python.framework/Versions/2.7/Python /path/to/my/venv/Python
$ ln -sf Python /path/to/my/venv/.Python
$ ln -sf /Library/Frameworks/Python.framework/Versions/2.7/Resources /path/to/my/venv/Resources

Your virtualenv structure would look something like:

lrwxr-xr-x   .Python@ -> Python
-rw-r--r--   .gitignore
-r-xr-xr-x   Python*
lrwxr-xr-x   Resources@ -> /Library/Frameworks/Python.framework/Versions/2.7/Resources
drwxr-xr-x   bin/
drwxr-xr-x   include/
drwxr-xr-x   lib/
-rw-r--r--   pyvenv.cfg

Keeping the Python dylib name the same and adding the Resources symlink seems to avoid breaking the code signature. (M1 macs may be stricter and require re-signing.)

NathanDunfield commented 2 years ago

Another workaround is to use the --copies flag

python2.7 -m virtualenv --copies /path/to/my/venv

Tested with Python.org's 2.7.18 and virtualenv==20.15.1 on macOS 12.4.

pypa / virtualenv

Virtualenv does not work correctly on python 2.7 from python.org and new macOS 12 Monterey #2284