search

teslamotors / ansible_puller

Ansible daemon for massively-scaleable Ansible
MIT License
125 stars 33 forks source link

virtualenv tool is failing when multiple python installations are installed #50

Closed sedillo closed 2 months ago

sedillo commented 2 months ago

Description

Currently makeVenv function relies on the virtualenv executable to create a virtual environment. However, this approach can lead to issues with specific Python versions and dependencies.

Current Behavior

Using virtualenv can lead to inconsistencies and errors, especially when using different Python versions. For example, with the virtualenv 20.0.1 (packaged with python 3.8) I get import errors when creating a python3.11 virtual environment:

$ virtualenv --version
virtualenv 20.0.17 from /usr/lib/python3/dist-packages/virtualenv/__init__.py

$ virtualenv --python /usr/bin/python3.11 test2
created virtual environment CPython3.11.9.final.0-64 in 67ms
  creator CPython3Posix(dest=/root/test2, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ ./test2/bin/pip list
Traceback (most recent call last):
  File "/root/./test2/bin/pip", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/commands/list.py", line 13, in <module>
    from pip._internal.cli.req_command import IndexGroupCommand
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 15, in <module>
    from pip._internal.index.package_finder import PackageFinder
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/index/package_finder.py", line 21, in <module>
    from pip._internal.index.collector import parse_links
  File "/root/test2/lib/python3.11/site-packages/pip/_internal/index/collector.py", line 12, in <module>
    from pip._vendor import html5lib, requests
ImportError: cannot import name 'html5lib' from 'pip._vendor' (/root/test2/lib/python3.11/site-packages/pip/_vendor/__init__.py)

In contrast, using the built-in venv module directly with Python has shown more consistent results:

$ /usr/bin/python3.11 -m venv test3
$ ./test3/bin/pip list
Package    Version
---------- -------
pip        24.0
setuptools 65.5.0

[notice] A new release of pip is available: 24.0 -> 24.1.2
[notice] To update, run: python3.11 -m pip install --upgrade pip

Proposed Change

Modify the makeVenv function to use the provided cfg.Python and cfg.Path parameters directly to create the virtual environment using the venv module.

cmd := exec.Command(cfg.Python, "-m", "venv", cfg.Path)
err := cmd.Run()
sedillo commented 2 months ago

This works now. Thanks.