search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Install Salt from the Salt package repositories here:
https://docs.saltproject.io/salt/install-guide/en/latest/
Apache License 2.0
14.19k stars 5.48k forks source link

[BUG] v3005 `config.items` produces a traceback #62849

Closed dafyddj closed 1 year ago

dafyddj commented 2 years ago

Description config.items appears broken in versions from 3005 onwards.

Setup Tested on Mac and Windows.

Please be as specific as possible and give set-up details.

Steps to Reproduce the behavior (Include debug logs if possible and relevant)

:~% salt-call config.items
/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.                                                                                
  warnings.warn("Setuptools is replacing distutils.")
[ERROR   ] Nested output failed:                       
Traceback (most recent call last):      
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/__init__.py", line 36, in try_printout                                                                                                     
    printout = get_printout(out, opts)(data, **kwargs)
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 149, in __call__                                                                                                            
    return self.loader.run(run_func, *args, **kwargs)   
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 1228, in run                                                                                                                
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)                                
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 1243, in _run_as                                                                                                            
    return _func_or_method(*args, **kwargs)                                                                      
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 155, in output                                                                                                            
    lines = nest.display(ret, base_indent, "", [])
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 143, in display                                                                                                           
    self.display(val, indent + 4, "", out)          
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 131, in display                                                                                                           
    keys = sorted(ret)                    
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/context.py", line 97, in __iter__                                                                                                          
    return self.value().__iter__()
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/context.py", line 72, in value                                                                                                             
    return loader.pack[self.name]                       
KeyError: '__opts__'                            

During handling of the above exception, another exception occurred:                                              

Traceback (most recent call last):
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/__init__.py", line 42, in try_printout                                                                                                     
    printout = get_printout("nested", opts)(data, **kwargs)                                                      
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 149, in __call__                                                                                                            
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 1228, in run                                                                                                                
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)                                
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/lazy.py", line 1243, in _run_as                                                                                                            
    return _func_or_method(*args, **kwargs)
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 155, in output
    lines = nest.display(ret, base_indent, "", [])
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 143, in display
    self.display(val, indent + 4, "", out)
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/output/nested.py", line 131, in display
    keys = sorted(ret)
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/context.py", line 97, in __iter__
    return self.value().__iter__()
  File "/usr/local/Cellar/salt/3005_1/libexec/lib/python3.10/site-packages/salt/loader/context.py", line 72, in value
    return loader.pack[self.name]
KeyError: '__opts__'
{'local': <salt.loader.context.NamedLoaderContext object at 0x10d3f8f10>}

Or run from the master produces this logging on the minion:

[DEBUG   ] LazyLoaded config.items
[INFO    ] Returning information for job: 20221010164558520073
[DEBUG   ] SaltEvent PUB socket URI: 4510
[DEBUG   ] SaltEvent PULL socket URI: 4511
[DEBUG   ] Closing IPCMessageClient instance
[ERROR   ] An un-handled exception from the multiprocessing process 'ProcessPayload(jid=20221010164558520073)' was caugh
t:
Traceback (most recent call last):
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\process.py", line 993, in wrapped_run_func
    return run_func()
  File "C:\Program Files\Salt Project\Salt\bin\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1835, in _target
    run_func(minion_instance, opts, data)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1829, in run_func
    return Minion._thread_return(minion_instance, opts, data)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 2050, in _thread_return
    minion_instance._return_pub(ret)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 2250, in _return_pub
    ret_val = self._send_req_sync(load, timeout=timeout)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1608, in _send_req_sync
    return event.fire_event(
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\event.py", line 813, in fire_event
    dump_data = salt.payload.dumps(data, use_bin_type=True)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\payload.py", line 173, in dumps
    return salt.utils.msgpack.packb(
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\msgpack.py", line 133, in packb
    return msgpack.packb(o, **_sanitize_msgpack_kwargs(kwargs))
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\msgpack\__init__.py", line 35, in packb
    return Packer(**kwargs).pack(o)
  File "msgpack\_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 298, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 295, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack\_packer.pyx", line 285, in msgpack._cmsgpack.Packer._pack
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\payload.py", line 168, in ext_type_encoder
    return dict(obj)
  File "C:\Program Files\Salt Project\Salt\bin\lib\_collections_abc.py", line 720, in __iter__
    yield from self._mapping
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\loader\context.py", line 97, in __iter__
    return self.value().__iter__()
AttributeError: 'NoneType' object has no attribute '__iter__'
Process ProcessPayload(jid=20221010164558520073):
Traceback (most recent call last):
  File "C:\Program Files\Salt Project\Salt\bin\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\process.py", line 993, in wrapped_run_func
    return run_func()
  File "C:\Program Files\Salt Project\Salt\bin\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1835, in _target
    run_func(minion_instance, opts, data)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1829, in run_func
    return Minion._thread_return(minion_instance, opts, data)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 2050, in _thread_return
    minion_instance._return_pub(ret)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 2250, in _return_pub
    ret_val = self._send_req_sync(load, timeout=timeout)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\minion.py", line 1608, in _send_req_sync
    return event.fire_event(
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\event.py", line 813, in fire_event
    dump_data = salt.payload.dumps(data, use_bin_type=True)
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\payload.py", line 173, in dumps
    return salt.utils.msgpack.packb(
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\utils\msgpack.py", line 133, in packb
    return msgpack.packb(o, **_sanitize_msgpack_kwargs(kwargs))
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\msgpack\__init__.py", line 35, in packb
    return Packer(**kwargs).pack(o)
  File "msgpack\_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 298, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 295, in msgpack._cmsgpack.Packer.pack
  File "msgpack\_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack\_packer.pyx", line 285, in msgpack._cmsgpack.Packer._pack
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\payload.py", line 168, in ext_type_encoder
    return dict(obj)
  File "C:\Program Files\Salt Project\Salt\bin\lib\_collections_abc.py", line 720, in __iter__
    yield from self._mapping
  File "C:\Program Files\Salt Project\Salt\bin\lib\site-packages\salt\loader\context.py", line 97, in __iter__
    return self.value().__iter__()
AttributeError: 'NoneType' object has no attribute '__iter__'

Expected behavior No traceback, and the config dictionary returned

Screenshots If applicable, add screenshots to help explain your problem.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml Salt Version: Salt: 3005 Dependency Versions: cffi: 1.14.6 cherrypy: 18.6.1 dateutil: 2.8.0 docker-py: Not Installed gitdb: 4.0.5 gitpython: 3.1.12 Jinja2: 3.1.0 libgit2: 1.5.0 M2Crypto: Not Installed Mako: 1.1.4 msgpack: 1.0.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: 2.21 pycrypto: 3.15.0 pycryptodome: 3.9.8 pygit2: 1.10.0 Python: 3.10.7 (main, Sep 15 2022, 01:52:03) [Clang 13.0.0 (clang-1300.0.29.30)] python-gnupg: 0.4.8 PyYAML: 5.4.1 PyZMQ: 23.2.0 smmap: 3.0.2 timelib: 0.2.4 Tornado: 4.5.3 ZMQ: 4.3.4 System Versions: dist: darwin 20.6.0 locale: utf-8 machine: x86_64 release: 20.6.0 system: Darwin version: 11.7 x86_64 ```

Additional context Add any other context about the problem here.

frebib commented 1 year ago

Does #63013 fix this?

whytewolf commented 1 year ago

looks like https://github.com/saltstack/salt/commit/02ad058a3325af0d58ee4de8809bd236955475a0 might have been a cause.

frebib commented 1 year ago

Reverting 02ad058 fixes it. It seems wrong to overload opts like that, to me. Edit: Applying #63013 also fixes this

eliasp commented 1 year ago

Fixed in current 3006-rc2 - IMHO this issue can be closed.