search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Install Salt from the Salt package repositories here:
https://docs.saltproject.io/salt/install-guide/en/latest/
Apache License 2.0
14.22k stars 5.49k forks source link

external_auth not functioning #34506

Closed akcrisp closed 6 years ago

akcrisp commented 8 years ago

Description of Issue/Question

Trying to configure external auth to use ldap(freeipa), running on centos7. for external authentication. I am a newbie to salt - so its possible I've missed a configuration step or misconfigured something...

Running salt-master -l trace, when i issue the salt -a ldap '*' test.ping from a logged in (ldap user account) on the master salt server all I get in the trace output is -

[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] IPCServer: Handling connection to address: 
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing

....

ldap itself is working fine - users can happily login ok. I've even tested disabling firewalld and selinux.

Setup

I have created auth.conf in master.d - which currently contains -

external_auth:
  ldap:
    saltusers%:
      - '*':
        - 'test.*'  

auth.ldap.server: <ipaserver.domainname>
auth.ldap.port: 389
auth.ldap.tls: False
auth.ldap.scope: 2
auth.ldap.uri: 'ldap://<ipaserver.domainname>:389'
auth.ldap.no_verify: False
auth.ldap.anonymous: False
auth.ldap.auth_by_group_membership_only: True
auth.ldap.groupou: groups
auth.ldap.groupclass: posixGroup
auth.ldap.accountattributename: memberUid
auth.ldap.basedn: cn=accounts,dc=domainname
auth.ldap.binddn: uid={{ username }},cn=users,cn=accounts,dc=localdomain
auth.ldap.minion_stripdomains: ['.localdomain']

Versions Report

Salt Version:
           Salt: 2016.3.1

Dependency Versions:
           cffi: 0.8.6
       cherrypy: 3.2.2
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.22.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

python-ldap-2.4.15-2.el7.x86_64 is installed - note ipa-client has this as a dependency

Ch3LL commented 8 years ago

@akcrisp Is there anything relevant in the master debug logs when restarting the master?

akcrisp commented 8 years ago

Can you give me a better idea what you mean by relevant ? Is there something I should be seeing when it starts and using external_auth? As other than see it say it's reading the auth.conf I don't see anything else related.

Andy

Sent from my iPhone

On 7 Jul 2016, at 18:47, Megan Wilhite notifications@github.com wrote:

@akcrisp Is there anything relevant in the master debug logs when restarting the master?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

akcrisp commented 8 years ago

High as I've not had a response for more info re what's relevant - please see the trace output from running the salt-master. This is unto the point its fully loaded. 

salt-master -l trace 
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[TRACE   ] None of the required configuration sections, 'logstash_udp_handler' and 'logstash_zmq_handler', were found in the configuration. Not loading the Logstash logging handlers module.
[TRACE   ] The required configuration section, 'fluent_handler', was not found the in the configuration. Not loading the fluent logging handlers module.
[DEBUG   ] Configuration file path: /etc/salt/master
[TRACE   ] Trying pysss.getgrouplist for 'root'
[TRACE   ] Trying generic group list for 'root'
[TRACE   ] Group list for user 'root': '[]'
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] Setting up the Salt Master
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[INFO    ] Preparing the root key for local communication
[DEBUG   ] Removing stale keyfile: /var/cache/salt/master/.root_key
[DEBUG   ] Created pidfile: /var/run/salt-master.pid
[INFO    ] The Salt Master is starting up
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Could not LazyLoad roots.init
[INFO    ] salt-master is starting as user 'root'
[INFO    ] Current values for max open files soft/hard setting: 1024/4096
[INFO    ] The value for the 'max_open_files' setting, 100000, is higher than what the user running salt is allowed to raise to, 4096. Defaulting to 4096.
[INFO    ] Raising max open files value to 4096
[INFO    ] New values for max open files soft/hard values: 4096/4096
[INFO    ] Creating master process manager
[INFO    ] Creating master publisher process
[DEBUG   ] Started 'salt.transport.zeromq.<type 'instancemethod'>._publish_daemon' with pid 5159
[INFO    ] Creating master event publisher process
[INFO    ] Starting the Salt Publisher on tcp://xxx.xxx.xxx.xxx:4505
[DEBUG   ] Started 'salt.utils.event.EventPublisher' with pid 5162
[INFO    ] Starting the Salt Puller on ipc:///var/run/salt/master/publish_pull.ipc
[TRACE   ] IPCMessagePublisher: binding to socket: /var/run/salt/master/master_event_pub.ipc
[TRACE   ] IPCServer: binding to socket: /var/run/salt/master/master_event_pull.ipc
[INFO    ] Creating master maintenance process
[DEBUG   ] Started 'salt.master.Maintenance' with pid 5163
[INFO    ] Creating master reactor process
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Error loading runners.nacl: libnacl import error, perhaps missing python libnacl package
[DEBUG   ] Could not LazyLoad timezone.get_offset
[DEBUG   ] Could not LazyLoad config.merge
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] Started 'salt.utils.reactor.Reactor' with pid 5256
[INFO    ] Creating master request server process
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] Started 'ReqServer' with pid 5257
[DEBUG   ] Process Manager starting!
[TRACE   ] Process manager iteration
[TRACE   ] IPCClient: Connecting to socket: /var/run/salt/master/master_event_pub.ipc
[TRACE   ] IPCServer: Handling connection to address: 
[DEBUG   ] Started 'salt.transport.zeromq.<type 'instancemethod'>.zmq_device' with pid 5261
[INFO    ] Setting up the master communication server
[DEBUG   ] Started 'MWorker-0' with pid 5276
[DEBUG   ] Started 'MWorker-1' with pid 5279
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Started 'MWorker-2' with pid 5280
[DEBUG   ] Started 'MWorker-3' with pid 5281
[DEBUG   ] Started 'MWorker-4' with pid 5282
[DEBUG   ] Process Manager starting!
[TRACE   ] Process manager iteration
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[DEBUG   ] Could not LazyLoad config.merge
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Updating roots fileserver cache
[DEBUG   ] This salt-master instance has accepted 1 minion keys.
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[TRACE   ] Loading core.zmqversion grain
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.get_server_id grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[TRACE   ] Loading core.ip6 grain
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Missing configuration file: /root/.saltrc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[TRACE   ] Loading core.hwaddr_interfaces grain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.ip4 grain
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: saltsrv.localdomain
[DEBUG   ] Reading configuration from /etc/salt/master
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[DEBUG   ] Including configuration from '/etc/salt/master.d/auth.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/auth.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/cherry.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/cherry.conf
[TRACE   ] Loading core.hwaddr_interfaces grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[TRACE   ] Loading core.hostname grain
[TRACE   ] Loading core.get_master grain
[TRACE   ] Loading core.pythonversion grain
[TRACE   ] Loading core.path grain
[TRACE   ] Loading core.get_server_id grain
[TRACE   ] Loading core.ip6 grain
[INFO    ] Worker binding to socket ipc:///var/run/salt/master/workers.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[TRACE   ] Loading core.ip4 grain
[TRACE   ] Loading core.saltversion grain
[TRACE   ] Loading core.saltpath grain
[TRACE   ] Loading core.pythonexecutable grain
[TRACE   ] Loading core.fqdn_ip4 grain
[TRACE   ] Loading core.dns grain
[TRACE   ] Loading core.fqdn_ip6 grain
[TRACE   ] Loading core.ip6_interfaces grain
[TRACE   ] Loading core.ip4_interfaces grain
[TRACE   ] Loading core.append_domain grain
[TRACE   ] Loading core.os_data grain
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[INFO    ] Worker binding to socket ipc:///var/run/salt/master/workers.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[INFO    ] Worker binding to socket ipc:///var/run/salt/master/workers.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[INFO    ] Worker binding to socket ipc:///var/run/salt/master/workers.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[TRACE   ] Loading core.zmqversion grain
[TRACE   ] Loading core.saltversioninfo grain
[TRACE   ] Loading core.pythonpath grain
[TRACE   ] Loading core.id_ grain
[TRACE   ] Loading core.locale_info grain
[TRACE   ] Loading core.get_machine_id grain
[TRACE   ] Loading core.ip_interfaces grain
[TRACE   ] Device sda reports itself as an HDD
[TRACE   ] Device sr0 reports itself as an HDD
[INFO    ] Worker binding to socket ipc:///var/run/salt/master/workers.ipc
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Loaded master key: /etc/salt/pki/master/master.pem
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing
[TRACE   ] _get_event() waited 5 seconds and received nothing
akcrisp commented 8 years ago

should add that the external_auth is all contained in the auth.conf, which you see it read

cro commented 8 years ago

Greetings @akcrisp,

I suspect that for some reason auth.conf isn't being loaded, because whenever anything goes wrong with eauth you should have messages in the log at the debug level and above. Do you get a different response if you temporarily move your ldap eauth configuration into /etc/salt/master ?

akcrisp commented 8 years ago

No it's exactly the same - tried it

Andy

On 12 Jul 2016, at 21:14, C. R. Oldham notifications@github.com wrote:

Greetings @akcrisp,

I suspect that for some reason auth.conf isn't being loaded, because whenever anything goes wrong with eauth you should have messages in the log at the debug level and above. Do you get a different response if you temporarily move your ldap eauth configuration into /etc/salt/master ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

rallytime commented 8 years ago

@akcrisp Do you get a sane result when trying a test.ping without going through the ldap user?

cro commented 8 years ago

Any change if you set auth.ldap.auth_by_group_membership_only: False?

akcrisp commented 8 years ago

Yeah that all works fine. And I have since tried pam rather than LDAP and that's not working even with it in the master config file either.

Sent from my iPhone

On 12 Jul 2016, at 21:23, Nicole Thomas notifications@github.com wrote:

@akcrisp Do you get a sane result when trying a test.ping without going through the ldap user?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

akcrisp commented 8 years ago

I will double check that but pretty certain I only added that later to see if it made any difference. It is as though it's ignoring entirely the external_auth

Sent from my iPhone

On 12 Jul 2016, at 21:30, C. R. Oldham notifications@github.com wrote:

Any change if you set auth.ldap.auth_by_group_membership_only: False?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

akcrisp commented 8 years ago

So I've put the external_auth back into the master file and I've set auth.ldap.auth_by_group_membership_only: False and its still exactly the same

Ch3LL commented 8 years ago

@akcrisp just wanted to clarify everything in this issue with you so far. Can you tell me if the following is true:

  1. We do not see anything in your debug logs about loading external_auth even when you are setting it to pam or ldap, therefore for some reason its not loading your external_auth settings from your master configuration file. Please correct me if that is incorrect. I went ahead and setup a test case I know absolutely works. Can you please try the following test case for setting up pam and report back whether it works for you or not.
  2. useradd saltdev
  3. passwd saltdev
  4. add the following to your /etc/salt/master:
external_auth:
  pam:
    saltdev:
      - 'ch3ll*':
        - test.*
  1. restart your salt-master (note: you should see LazyLoaded pam.auth in your master debug log file)
  2. then attempt to run test.ping against your minion
[ch3ll@thecakeisalie ~]$ sudo salt -a pam 'ch3ll-minion02' test.ping 
username: saltdev
password: 
ch3ll-minion02:
    True

Does the above test case work in your environment?

akcrisp commented 8 years ago

Hi Megan we have completed our evaluation now and passed our findings over to the engineering function who will include it in their decision making progress.

Thanks for coming back to me.

Regards

Andy

Sent from my iPhone

On 10 Aug 2016, at 23:41, Megan Wilhite notifications@github.com wrote:

@akcrisp just wanted to clarify everything in this issue with you so far. Can you tell me if the following is true:

We do not see anything in your debug logs about loading external_auth even when you are setting it to pam or ldap, therefore for some reason its not loading your external_auth settings from your master configuration file. Please correct me if that is incorrect. I went ahead and setup a test case I know absolutely works. Can you please try the following test case for setting up pam and report back whether it works for you or not.

useradd saltdev

passwd saltdev add the following to your /etc/salt/master: external_auth: pam: saltdev:

  • 'ch3ll_':
    • test._ restart your salt-master (note: you should see LazyLoaded pam.auth in your master debug log file) then attempt to run test.ping against your minion [ch3ll@thecakeisalie ~]$ sudo salt -a pam 'ch3ll-minion02' test.ping username: saltdev password: ch3ll-minion02: True Does the above test case work in your environment?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

meggiebot commented 8 years ago

@akcrisp thanks for the update. If you have a chance to provide the information we requested it will help us determine the severity and priority of this issue as well as if it's impacting other Salt users.

cinterloper commented 8 years ago

I believe I am also, currently experiencing this issue.

also, i only get the _get_event() waited 5 seconds and received nothing message on salt-api -l trace when my program is connected to the cherrypy websocket I am using salt version salt-master 2016.3.2 (Boron)

after rolling back to salt 2015.8.8 (Beryllium) everything behaves as expected

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.