How to overwrite the default host dict

[rmatulat]

Hi, I need to disable the IPv6 entries in /etc/hosts like they are defined at the defaults.yaml. So we need a dict like this:

  hosts:
    def_entries:
      - name: localhost
        ip: 127.0.0.1
      - name: localhost.localdomain
        ip: 127.0.0.1

I am unable to overwrite the default values with a pillar because it looks like the key of a pillar hosts:def_entries is added to hosts:def_entries of the defaults.yaml.

I made a pillar like this:

network:
  lookup:
    hosts:
      def_entries:
        - name: localhost
          ip: 127.0.0.1
        - name: localhost.localdomain
          ip: 127.0.0.1

This leads to a datastructure hosts inside your hosts.sls like this:

yaml = [{ip: 127.0.0.1, name: localhost}, {ip: 127.0.0.1, name: localhost.localdomain}, {
    ip: '::1', name: localhost}, {ip: '::1', name: ip6-localhost}, {ip: '::1', name: ip6-loopback},
  {ip: 'fe00::0', name: ip6-localnet}, {ip: 'ff00::0', name: ip6-mcastprefix}, {ip: 'ff02::1',
    name: ip6-allnodes}, {ip: 'ff02::2', name: ip6-allrouters}, {ip: 'ff02::3', name: ip6-allhosts},
  {ip: 127.0.0.1, name: localhost}, {ip: 127.0.0.1, name: localhost.localdomain}]

As you can see, there are 2 {ip: 127.0.0.1, name: localhost} entries.

Any idea how to fix this? Kind regards ralf

[rmatulat]

I guess the main problem relies on how salt.utils.dictupdate.update() is doing the merge. As long as in a key:value dict value is a string, it will be overwritten as expected. If value is a list the lists will be added together. So thats why we get doubled entries as discribed above.

I wonder why no one came across this issue before.

[arnisoph]

Thanks, I will see what's possible!

[arnisoph]

@rall0r in the short-term you can simply set def_entries to an empty list [] like in https://github.com/bechtoldt/saltstack-network-formula/blob/master/pillar.example.sls#L7 and explictly define all host entries like in https://github.com/bechtoldt/saltstack-network-formula/blob/master/pillar.example.sls#L71

Does it resolve your issue?

[arnisoph]

I think this is related to the new strategy that salt merges now lists instead of overwriting them. This will require some more work on (almost) all of my formulas.

[rmatulat]

We tried to overwrite def_entries, that was our first aproach. Then we realized, that this does not overwrite def_entries but just merges the defaults.yaml - def_entries and the lookup def_entries together. I guess I have to write some proof of concept state/pillar and open an issue at saltstack itself. Nevertheless it would be great if you would come up with some solution :-)

[arnisoph]

For reference:

• https://github.com/saltstack/salt/issues/21313
• https://github.com/saltstack/salt/pull/25358
• https://github.com/saltstack/salt/issues/25954
• https://github.com/saltstack/salt/issues/27890

[rmatulat]

I read your interesting references. That shed some light on that whole "merge" topic. We are using pillar merges quite heavily, so we have to keep an eye on it. But: In case of grains.filter_by merge issues the doc is rather explicit: https://github.com/saltstack/salt/blob/develop/salt/modules/grains.py#L468

Of course this is not something in your responsibility. I guess I'll have to investigate a bit further and see if someone already came up somewhere with the same issues and if not to provide some testcase.

[arnisoph]

I will update the network formula to my (more or less) new formula structure (like https://github.com/bechtoldt/saltstack-elasticsearch-formula) that uses https://github.com/bechtoldt/salt-modules/blob/master/_modules/formhelper.py which gives us more flexibility when merging pillar with defaults.yaml.

This of course will require users to modify their way how they use my old-styled formulas, but the structure/layout/design/whatever has proven to be very useful for more people.