custom python install breaks ansible and molecule -- get_system_sitepackages()

[br-olf]

Hello, this issue is related to #58 and #17

I encountered an error when my systems python version upgraded (to 3.11) and I still need an older one (3.10) for molecule. So i installed it from source.

But all ansible commands broke with the following error:

Traceback (most recent call last):
  File "/path/to/my/venv/bin/ansible-config", line 5, in <module>
    from ansible.cli.config import main
  File "/path/to/my/venv/lib/python3.10/site-packages/ansible/cli/__init__.py", line 101, in <module>
    from ansible.module_utils.common.file import is_executable
  File "/path/to/my/venv/lib/python3.10/site-packages/ansible/module_utils/common/file.py", line 25, in <module>
    import selinux
  File "/path/to/my/venv/lib/python3.10/site-packages/selinux/__init__.py", line 106, in <module>
    check_system_sitepackages()
  File "/path/to/my/venv/lib/python3.10/site-packages/selinux/__init__.py", line 95, in check_system_sitepackages
    system_sitepackages = get_system_sitepackages()
  File "/path/to/my/venv/lib/python3.10/site-packages/selinux/__init__.py", line 81, in get_system_sitepackages
    subprocess.check_output(
  File "/usr/local/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.10/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/python3.10'

After tedious hours of bug hunting I discovered the following code in selinux/__init__.py:

    def get_system_sitepackages():
        """Get sitepackage locations from system python"""
        # Do not ever use sys.executable here
        # See https://github.com/pycontribs/selinux/issues/17 for details
        system_python = "/usr/bin/python%s" % ".".join(
            [str(item) for item in platform.python_version_tuple()[0:2]]
        )

        system_sitepackages = json.loads(
            subprocess.check_output(
                [
                    system_python,
                    "-c",
                    "import json, site; print(json.dumps(site.getsitepackages()))",
                ]
            ).decode("utf-8")
        )
        return system_sitepackages

That code assumes that any installed python is always located under /usr/bin which is problematic.

I don't know what was the reason for not using sys.executable but after seeing the original issue was related to python2 I think it will work nowadays.

I made a pull request and didn't found any issues in my tests: https://github.com/pycontribs/selinux/pull/68

[joshtrutwin]

+1

cannot use pyenv etc - thanks!

[minsis]

This issue is also problematic for conda environments.

[cinnion]

This problem also occurs if you are using python built from source, which installs into /usr/local. A hacky workaround is to create a symlink to the actual python executable being used under /usr/bin, but the better fix would be to once again revisit #17 and once again see what was wrong with using sys.executable.

Specifics: My system has Python 3.10 installed in /usr/local/bin/python3.10 and is being invoked as python3.10. get_system_sitepackages() is looking for the executable in /usr/bin, but depending on the system, it could be /usr/local/bin, or even somewhere under a directory such as /opt (such as might be the case for a RHEL system where it is installed via SCL). sys.executable returns the correct path of '/usr/local/bin/python3.10', and if I use the python3 symlink under /usr/local/bin, it returns the path to that symlink, which the OS would redirect properly.