Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
Please be as specific as possible and give set-up details.
[ ] on-prem machine
[x] VM - Virtualbox
[ ] VM running on a cloud service, please be explicit and add details
[ ] container (Kubernetes, Docker, containerd, etc. please specify)
[ ] or a combination, please be explicit
[ ] jails if it is FreeBSD
[ ] classic packaging
[ ] onedir packaging
[ ] used bootstrap to install
Steps to Reproduce the behavior
Setting ipc_mode: tcp is causing that after clean start of service (after reboot) are socket files not created:
root@sm-3007:~# ls -la /var/run/salt/master/
total 0
drwxr-xr-x 2 root root 40 Jun 14 10:21 .
drwxr-xr-x 4 root root 80 Jun 14 08:01 ..
If these files are missing, salt-api is not able to talk to Salt Master service, and therefor all salt-api calls are getting 503 ..
user@sm-3007:~$ salt-pepper '*' test.version
Error with request: HTTP Error 503: Service Unavailable
ERROR 2024-06-14 09:59:04,940 libpepper: Error with request: HTTP Error 503: Service Unavailable
HTTP Error 503: Service Unavailable
Uncaught Pepper error (increase verbosity for the full traceback).
server side:
[INFO ] [14/Jun/2024:09:58:51] ENGINE Serving on https://0.0.0.0:8000
[14/Jun/2024:09:58:51] ENGINE Bus STARTED
[INFO ] [14/Jun/2024:09:58:51] ENGINE Bus STARTED
127.0.0.1 - - [14/Jun/2024:09:59:04] "POST /login HTTP/1.1" 503 2184 "" "Python-urllib/3.10"
[INFO ] 127.0.0.1 - - [14/Jun/2024:09:59:04] "POST /login HTTP/1.1" 503 2184 "" "Python-urllib/3.10"
In case that ipc_mode is not set to tcp, so it has default value ipc, so salt-api is working properly, but it's breaking functionality of external authentication..
root@sm-3007:~# grep ipc /etc/salt/master.d/external_auth.conf
#ipc_mode: tcp
root@sm-3007:~# systemctl restart salt-master.service salt-minion
root@sm-3007:~# ls -la /var/run/salt/master/
total 0
drwxr-xr-x 2 root root 120 Jun 14 10:13 .
drwxr-xr-x 4 root root 80 Jun 14 08:01 ..
srw------- 1 root root 0 Jun 14 10:13 master_event_pub.ipc
srw------- 1 root root 0 Jun 14 10:13 master_event_pull.ipc
srw------- 1 root root 0 Jun 14 10:13 publish_pull.ipc
srw------- 1 root root 0 Jun 14 10:13 workers.ipc
user@sm-3007:~$ salt -C '*' test.ping
password:
[WARNING ] TCP Publish Client encountered an exception while connecting to /var/run/salt/master/master_event_pub.ipc: StreamClosedError('Stream is closed'), will reconnect in 1 seconds - File "/usr/bin/salt", line 11, in <module>
sys.exit(salt_main())
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/scripts.py", line 528, in salt_main
client.run()
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/cli/salt.py", line 192, in run
for full_ret in cmd_func(**kwargs):
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 815, in cmd_cli
self.pub_data = self.run_job(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 387, in run_job
pub_data = self.pub(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1904, in pub
if listen and not self.event.connect_pub(timeout=timeout):
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/event.py", line 323, in connect_pub
self.subscriber = salt.utils.asynchronous.SyncWrapper(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 76, in __init__
self.obj = cls(*args, **kwargs)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/base.py", line 210, in ipc_publish_client
return publish_client(opts, io_loop, **kwargs)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/base.py", line 152, in publish_client
return salt.transport.tcp.PublishClient(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/tcp.py", line 220, in __init__
super().__init__(opts, io_loop, **kwargs)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/base.py", line 398, in __init__
super().__init__()
[ERROR ] An un-handled exception was caught by Salt's global exception handler:
TypeError: argument must be an int, or have a fileno() method.
Traceback (most recent call last):
File "/usr/bin/salt", line 11, in <module>
sys.exit(salt_main())
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/scripts.py", line 528, in salt_main
client.run()
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/cli/salt.py", line 192, in run
for full_ret in cmd_func(**kwargs):
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 830, in cmd_cli
for fn_ret in self.get_cli_event_returns(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1653, in get_cli_event_returns
for ret in self.get_iter_returns(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1187, in get_iter_returns
for raw in ret_iter:
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1104, in get_returns_no_block
raw = self.event.get_event(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/event.py", line 651, in get_event
ret = self._get_event(wait, tag, match_func, no_block)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/event.py", line 553, in _get_event
raw = self.subscriber.recv(timeout=wait)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 138, in wrap
raise exc_info[1].with_traceback(exc_info[2])
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 146, in _target
result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
File "/opt/saltstack/salt/lib/python3.10/site-packages/tornado/ioloop.py", line 527, in run_sync
return future_cell[0].result()
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/tcp.py", line 375, in recv
events, _, _ = select.select([self._stream.socket], [], [], 0)
TypeError: argument must be an int, or have a fileno() method.
Traceback (most recent call last):
File "/usr/bin/salt", line 11, in <module>
sys.exit(salt_main())
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/scripts.py", line 528, in salt_main
client.run()
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/cli/salt.py", line 192, in run
for full_ret in cmd_func(**kwargs):
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 830, in cmd_cli
for fn_ret in self.get_cli_event_returns(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1653, in get_cli_event_returns
for ret in self.get_iter_returns(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1187, in get_iter_returns
for raw in ret_iter:
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/__init__.py", line 1104, in get_returns_no_block
raw = self.event.get_event(
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/event.py", line 651, in get_event
ret = self._get_event(wait, tag, match_func, no_block)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/event.py", line 553, in _get_event
raw = self.subscriber.recv(timeout=wait)
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 138, in wrap
raise exc_info[1].with_traceback(exc_info[2])
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 146, in _target
result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
File "/opt/saltstack/salt/lib/python3.10/site-packages/tornado/ioloop.py", line 527, in run_sync
return future_cell[0].result()
File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/tcp.py", line 375, in recv
events, _, _ = select.select([self._stream.socket], [], [], 0)
TypeError: argument must be an int, or have a fileno() method.
.. which is pointing to permissions issues.. so after adding rw permissions for others to /var/run/salt/master/master_event_pub.ipc which I am aware that is REALLY STUPID IDEA.. and I wouldn't mention this aloud in decent society.
root@sm-3007:~# chmod o+rw /var/run/salt/master/master_event_pub.ipc
root@sm-3007:~# ls -la /var/run/salt/master/master_event_pub.ipc
srw----rw- 1 root root 0 Jun 14 10:13 /var/run/salt/master/master_event_pub.ipc
root@sm-3007:~# su - user
user@sm-3007:~$ salt -C '*' test.ping
sm-3007:
True
Expected behavior
I would expect that external authentication against pam or ldap is not mutually exclusive with using salt-api. So there is possibility to use both features at same time.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
```yaml
root@sm-3007:~# salt --versions-report
Salt Version:
Salt: 3007.1
Python Version:
Python: 3.10.14 (main, Apr 3 2024, 21:30:09) [GCC 11.2.0]
Dependency Versions:
cffi: 1.16.0
cherrypy: unknown
dateutil: 2.8.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.4
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 23.1
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.16.0
smmap: Not Installed
timelib: 0.3.0
Tornado: 6.3.3
ZMQ: 4.3.4
Salt Package Information:
Package Type: onedir
System Versions:
dist: ubuntu 22.04.4 jammy
locale: utf-8
machine: x86_64
release: 5.15.0-107-generic
system: Linux
version: Ubuntu 22.04.4 jammy
```
Additional context
Run user is set to root because of intend to authenticate locally using pam module.
Description In Salt
3007.1external_authrequiresipc_mode: tcpwhich is breaking functionality ofsalt-apiI need to setup
external_authon Salt Master3007.1, which is not working properly without settingipc_mode: tcpas I've found in this blog.If it's intended change, it would be great if official documentation was also updated!
Setup Salt Master configuration:
/etc/salt/master.d/run_user.conf/etc/salt/master.d/external_auth.conf/etc/salt/master.d/cherrypy_ssl.confPlease be as specific as possible and give set-up details.
Steps to Reproduce the behavior Setting
ipc_mode: tcpis causing that after clean start of service (after reboot) are socket files not created:If these files are missing,
salt-apiis not able to talk to Salt Master service, and therefor allsalt-apicalls are getting503..server side:
In case that
ipc_modeis not set totcp, so it has default valueipc, sosalt-apiis working properly, but it's breaking functionality of external authentication..salt-apicall usingpepper..but run using external authentication module:
.. which is pointing to permissions issues.. so after adding
rwpermissions for others to/var/run/salt/master/master_event_pub.ipcwhich I am aware that is REALLY STUPID IDEA.. and I wouldn't mention this aloud in decent society.Expected behavior I would expect that external authentication against pam or ldap is not mutually exclusive with using
salt-api. So there is possibility to use both features at same time.Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml root@sm-3007:~# salt --versions-report Salt Version: Salt: 3007.1 Python Version: Python: 3.10.14 (main, Apr 3 2024, 21:30:09) [GCC 11.2.0] Dependency Versions: cffi: 1.16.0 cherrypy: unknown dateutil: 2.8.2 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.4 libgit2: Not Installed looseversion: 1.3.0 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.7 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 23.1 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: Not Installed python-gnupg: 0.5.2 PyYAML: 6.0.1 PyZMQ: 25.1.2 relenv: 0.16.0 smmap: Not Installed timelib: 0.3.0 Tornado: 6.3.3 ZMQ: 4.3.4 Salt Package Information: Package Type: onedir System Versions: dist: ubuntu 22.04.4 jammy locale: utf-8 machine: x86_64 release: 5.15.0-107-generic system: Linux version: Ubuntu 22.04.4 jammy ```Additional context Run user is set to
rootbecause of intend to authenticate locally usingpammodule.