[BUG] pip.installed docker causes swarm.swarm_init to throw errors

SammyTbeile commented 4 years ago

Description of Issue

When using pip.installed to install docker python sdk for use with the swarm module, calling the swarm.swarm_init (or any other swam function) in a module.run during this execution will fail. The states will both succeed if run in separate runs but not if they're in the same highstate (whether from the same sls file or different sls files).

Setup

pip_install_docker:
  pip.installed:
    - name: docker
start_swarm:
  module.run:
    - swarm.swarm_init:
      - advertise_addr: {{ salt['grains.get']('ipv4')[0] }}
      - listen_addr: '0.0.0.0'
      - force_new_cluster: False

Steps to Reproduce Issue

Running the above states (either as one sls or two) in the same run:

----------
          ID: pip_install_docker
    Function: pip.installed
        Name: docker
      Result: True
     Comment: Python package docker was already installed
              All specified packages are already installed
     Started: 11:02:24.950155
    Duration: 1494.427 ms
     Changes:
----------
          ID: start_swarm
    Function: module.run
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1951, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 631, in _decorate
                  return self._call_function(kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 334, in _call_function
                  raise error
              HTTPError: 503 Server Error: Service Unavailable for url: http+docker://localhost/v1.35/swarm
     Started: 11:02:26.448862
    Duration: 6.435 ms
     Changes:

To view the expected behavior, apply a sls file containing only the first state

          ID: install_docker_python_sdk
    Function: pip.installed
        Name: docker
      Result: True
     Comment: Python package docker was already installed
              All specified packages are already installed
     Started: 11:09:08.521908
    Duration: 1488.641 ms
     Changes:

And then another containing just the second:

                   ID: initialize_new_swarm
    Function: module.run
      Result: True
     Comment: swarm.swarm_init: Success
     Started: 11:10:49.592136
    Duration: 668.464 ms
     Changes:
              ----------
              swarm.swarm_init:
                  ----------
                  Comment:
                      Docker swarm has been initialized on  and the worker/manager Join token is below
                  Tokens:

Versions Report

Master:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
      dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: 0.26.8
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.17
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.16 (default, Apr  6 2019, 01:42:57)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.2
           RAET: Not Installed
          smmap: 2.0.5
        timelib: Not Installed
        Tornado: 5.1.1
            ZMQ: 4.3.1

System Versions:
           dist: debian 10.0
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-5-amd64
         system: Linux
        version: debian 10.0

Minion:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed

     dateutil: 2.5.3
      docker-py: 4.1.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1

System Versions:
           dist: debian 9.9
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-9-amd64
         system: Linux
        version: debian 9.9

SammyTbeile commented 4 years ago

I think this is another instance of a well documented issue: https://github.com/saltstack/salt/issues/24925

SammyTbeile commented 4 years ago

Also, for any searchers that end up here before this is fixed I found a workaround where if you pip.install docker in a salt-call instead of via the state, it works.

pip_install_docker:
  cmd.run:
    - name: "salt-call pip.install docker"

saltstack / salt