search

saltstack / relenv

Re-producible and Re-relocatable Python Environments
Apache License 2.0
25 stars 16 forks source link

Installing packages with Salt-Pip fails: JSONDecodeError #115

Closed vps-eric closed 1 year ago

vps-eric commented 1 year ago

Installing packages with salt-pip (Salt 3006.0) fails with a JSONDecodeError and what appears to be some recursive behavior.

To reproduce, set up a fresh Almalinux 8.7 box (I use Vagrant on QEMU) and manually install the Salt 3006.0 minion. Add a symlink according to #114:

ln -s /opt/saltstack/salt/bin/python3 /usr/bin/python3

Then, attempt to install pycryptodome (or any other package in my experience):

/opt/saltstack/salt/salt-pip install pycryptodome
Stack Trace ```shell Traceback (most recent call last): File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/__main__.py", line 29, in from pip._internal.cli.main import main as _main File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 9, in from pip._internal.cli.autocompletion import autocomplete File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/autocompletion.py", line 10, in from pip._internal.cli.main_parser import create_main_parser File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main_parser.py", line 9, in from pip._internal.build_env import get_runnable_pip File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/build_env.py", line 20, in from pip._internal.cli.spinners import open_spinner File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/spinners.py", line 9, in from pip._internal.utils.logging import get_indentation File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/logging.py", line 29, in from pip._internal.utils.misc import ensure_dir File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 42, in from pip._internal.locations import get_major_minor_version File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/__init__.py", line 14, in from . import _sysconfig File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/_sysconfig.py", line 11, in from .base import change_root, get_major_minor_version, is_osx_framework File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/base.py", line 16, in site_packages: typing.Optional[str] = sysconfig.get_path("purelib") File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path return get_paths(scheme, vars, expand)[name] File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped paths = func(scheme=scheme, vars=vars, expand=expand) File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths return _expand_vars(scheme, vars) File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars _extend_dict(vars, get_config_vars()) File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 131, in wrapped _CONFIG_VARS = func() File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 640, in get_config_vars srcdir = os.path.dirname(get_makefile_filename()) File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 399, in get_makefile_filename return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile') File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path return get_paths(scheme, vars, expand)[name] File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped paths = func(scheme=scheme, vars=vars, expand=expand) File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths return _expand_vars(scheme, vars) File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars _extend_dict(vars, get_config_vars()) File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 140, in wrapped _SYSTEM_CONFIG_VARS = json.loads(p.stdout[:-1]) File "/opt/saltstack/salt/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/opt/saltstack/salt/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/opt/saltstack/salt/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ```

The JSONDecodeError is due to get_config_vars_wrapper(...) parsing the stdout of a subprocess Python call to sysconfig: in my environment, calling sysconfig.get_config_vars() actually prints nothing (stdout empty), causing JSON decoding to fail. (Side note: did Rube Goldberg write this function? I'm sure there's a reason for the way this is implemented; could someone elaborate?)

Digging deeper into sysconfig.get_config_vars() reveals some apparently recursive behavior. The following is what I have observed in my environment when calling sysconfig.get_config_vars() in the same manner as get_config_vars_wrapper() from Salt 3006.0's Python 3.10 REPL:

  1. sysconfig.get_config_vars() is called without arguments. _CONFIG_VARS is None, the os.name is posix and _PYTHON_BUILD is falsy. Execution continues to get_makefile_filename().
  2. Execution continues to a call to get_path('stdlib').
  3. get_paths('posix_prefix', None, True) is called. It in turn calls _expand_vars('posix_prefix', None).
  4. _expand_vars calls _extend_dict(None, get_config_vars()). Notice that get_config_vars() is now being recursively called.
  5. At this point in practice, the wrapper is called instead of get_config_vars. The wrapper, again, calls sysconfig.get_config_vars(None) within a new Python subprocess, and this time _CONFIG_VARS has a bunch (722 for me) of variables, which it returns.
  6. For some reason I haven't wrapped my mind around, the now non-empty variables result is not returned from the wrapper back to _extend_dict. In fact, _extend_dict is never called. Instead, the wrapper recursively calls itself. I've seen it do so up to 1,024 times, and have heard from another user with this problem that it is accompanied by a spike in memory and crash of the Salt minion and master.

Interestingly, this behavior seems to only occur when using Salt's bundled Python at /usr/bin/python3. Compiling Python 3.10.11 from source and adding the appropriate symlink for that scenario (ln -s /usr/local/bin/python3 /usr/bin/python3) does not present this problem.

vps-eric commented 1 year ago

See #114; Python should be installed on the system.