search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.11k stars 5.47k forks source link

Can't reuse minion_id, after deleting ane accepting key [BUG] #57183

Open jeffdyke opened 4 years ago

jeffdyke commented 4 years ago

Description I would like to destroy a node/ec2 instance and create another with the same name, but there seems to be a cache issue somewhere due to the following (i posted this in r/saltstack, so jut copy and pasing)

I had been rebuilding a couple of webservers using terraform, so i taint the instance, and rebuild it. Before i start rebuilding i:

I see this will remove it from /etc/salt/pki/master/minions/ and adding the key back, adds a new file.

Results in 

The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/minion.py", line 1676, in _thread_return
        return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
      File "/usr/lib/python3/dist-packages/salt/executors/direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 1087, in highstate
        orchestration_jid=orchestration_jid)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 4088, in call_highstate
        return self.state.call_high(high, orchestration_jid)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 2896, in call_high
        high, req_in_errors = self.requisite_in(high)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1652, in requisite_in
        in iter(high[_id])
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1653, in <listcomp>
        if not state.startswith('__')]:
    AttributeError: 'OrderedDict' object has no attribute 'startswith'

If i change the /etc/salt/minion_id to to_be_resusedA and restart, state.highstate runs perfectly.

I've been building machines with saltstack for years. I'm surprised i've not come across t his before, but perhaps something changed.

Thanks for any cache busting pointers, thought i'd post it not on the issues board first b/c i think i'm just missing a step. Setup I only override the hash_type, which is likely no longer required. I don't think it has to do with setup.

Steps to Reproduce the behavior Create a minion, that listens to a master, with a minion_id or foo, everything thing else is in the description. If i destroy foo, remove the key from salt, create a new foo add the key to salt...that causes the problem on highstate.

Expected behavior For salt-key -d foo -y to clear everything the salt master knows about it. So foo can be used again.

Screenshots

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) minion prodweb01 ``` Salt Version: Salt: 3000.2 Dependency Versions: cffi: 1.11.5 cherrypy: Not Installed dateutil: 2.8.1 docker-py: 4.2.0 gitdb: 2.0.0 gitpython: 2.1.3 Jinja2: 2.9.6 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack-pure: Not Installed msgpack-python: 0.4.8 mysql-python: Not Installed pycparser: 2.18 pycrypto: 3.7.3 pycryptodome: Not Installed pygit2: Not Installed Python: 3.6.9 (default, Apr 18 2020, 01:56:04) python-gnupg: 0.4.1 PyYAML: 3.13 PyZMQ: 16.0.2 smmap: 2.0.5 timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.2.5 System Versions: dist: Ubuntu 18.04 bionic locale: UTF-8 machine: x86_64 release: 5.3.0-1017-aws system: Linux version: Ubuntu 18.04 bionic ``` Master Config ``` Salt Version: Salt: 3000.2 Dependency Versions: cffi: 1.11.5 cherrypy: Not Installed dateutil: 2.7.5 docker-py: Not Installed gitdb: 2.0.0 gitpython: 2.1.3 Jinja2: 2.9.6 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack-pure: Not Installed msgpack-python: 0.4.8 mysql-python: Not Installed pycparser: 2.18 pycrypto: 3.7.3 pycryptodome: Not Installed pygit2: Not Installed Python: 3.7.5 (default, Nov 7 2019, 10:50:52) python-gnupg: 0.4.1 PyYAML: 5.1.2 PyZMQ: 18.1.0 smmap: 2.0.5 timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.3.2 System Versions: dist: Ubuntu 18.04 bionic locale: UTF-8 machine: x86_64 release: 5.3.0-1017-aws system: Linux version: Ubuntu 18.04 bionic ```

Additional context It seems like this is a simple cache issue, b/c a new minion_id always works.

jeffdyke commented 4 years ago

I can't stress enough how much i think this is just a cache issue. If i'm wrong, hopefully you have enough data to consider what it may be. Also, on another point, some of these minions have been up for years on the same master, constantly upgraded. So i don't really know if you could simply recreate, this could have to do with older data???