host.present produces duplicates

[quantum-x]

host.present creates duplicates in the /etc/hosts:

Source:

  host.present:
    - ip: 173.246.35.212

mysqlSlave:
  host.present:
    - ip: 173.246.35.212

memcachedServer:
  host.present:
    - ip: 173.246.35.212

sphinxServer:
  host.present:
    - ip: 173.246.35.218

Result after a few updates

192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer
192.168.122.108         sphinxServer
192.168.122.108         memcachedServer mysqlMaster     mysqlSlave
192.168.122.108         sphinxServer

           Salt: 0.15.1
         Python: 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
         Jinja2: unknown
       M2Crypto: 0.20.2
 msgpack-python: 0.1.9.final
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
         PyYAML: 3.10
          PyZMQ: 2.2.0.1
            ZMQ: 3.2.3

[zhangcheng]

Suggest to provide the output of salt --versions-report

[quantum-x]

Apologies, slipped my mind:

           Salt: 0.15.1
         Python: 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
         Jinja2: unknown
       M2Crypto: 0.20.2
 msgpack-python: 0.1.9.final
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
         PyYAML: 3.10
          PyZMQ: 2.2.0.1
            ZMQ: 3.2.3

[basepi]

That's no good! Thanks for the report, we'll look into it.

[terminalmage]

@quantum-x it looks like you're missing a line (or more) at the top of your SLS example. Additionally, I noticed that the IP address that keeps repeating is not one of the ones defined in your SLS. Did entries for this IP address and those hosts exist prior to running your state?

[quantum-x]

Total config is as such:

database.sls

db-packages:
  pkg.installed:
    - names:
      - mysql-server
      - mysqltuner
      - memcached
      - MySQL-python
      - sphinx

  service:
    - name: mysqld
    - running

  service:
    - name: memcached
    - running

  service:
    - name: searchd
    - running

/root/sql:
  file.recurse:
    - source: salt://prod-db/sql/
    - include_empty: True
    - clean: True

/etc/salt/minion.d/mysql.conf:
  file.managed:
    - source: salt://prod-db/etc/salt/minion.d/mysql.conf

#Includes the hosts tweaks
include:
  - hosts

hosts.sls

mysqlMaster:
  host.present:
    - ip: 173.246.35.212

mysqlSlave:
  host.present:
    - ip: 173.246.35.212

memcachedServer:
  host.present:
    - ip: 173.246.35.212

sphinxServer:
  host.present:
    - ip: 173.246.35.218

The IPs are different simply because I pasted the results on the staging server (sed replaces the IPs w/ staging ones) - because I had to remove the excess lines from the production environment.

Apologies if this is a stupid user (me) error - but I can replicate the issue.

[terminalmage]

OK, so are you saying that there is a separate but similar SLS file being used in staging, in which the IPs are the one that matches the example you initially posted?

I'm trying to get a handle on whether or not entries for different IPs than the ones in the SLS already existed prior to Salt repeatedly appending duplicates.

[quantum-x]

Hi - sorry for the confusion. In my staging deployment script, there's a line that runs: salt -N $cluster cmd.run "sed -i 's/173.246.35.212/192.168.122.108/' /etc/hosts"

The only reason that the IPs in the dump above are different is because the above line modifies them after the fact. Apologies for overlooking this when I first posted - as I said, I'd yanked the repititions out of production as it was breaking the app.

[terminalmage]

OK, then this really isn't a bug. Salt's support for host files builds a dictionary of IPs and the hosts that are associated with each IP. The fact that you keep changing the IPs is what is causing the value to continue to be added, since Salt does not find host sphinxServer associated with IP 173.246.35.218.

If you need different IPs for different environments, then you should perhaps consider storing the hosts and IPs in Pillar and then use a jinja loop to create the appropriate host.present states.

[quantum-x]

Hi Erik - OK, thanks - I understand where the error manifests from now. Thanks kindly for your assistance, and apologies for any trouble caused.

On Wed, Jun 19, 2013 at 7:24 PM, Erik Johnson notifications@github.comwrote:

OK, then this really isn't a bug. Salt's support for host files builds a dictionary of IPs and the hosts that are associated with each IP. The fact that you keep changing the IPs is what is causing the value to continue to be added.

If you need different IPs for different environments, then you should perhaps consider storing the hosts and IPs in Pillar and then use a jinja loop to create the appropriate host.present states.

— Reply to this email directly or view it on GitHubhttps://github.com/saltstack/salt/issues/5583#issuecomment-19699355 .

[terminalmage]

Oh, no trouble at all. If you want to see a little more about what I was talking about, you can run the hosts.list_hosts command from the CLI, and you'll see how Salt compiles the list of the IPs and hosts. It is that list against which your states are compared.

I'll go ahead and close this report.