[not-really-a-bug] Task execution order for custom config?

[ssasso]

Given this topology extract:

plugin: [ bgp.session, bgp.policy ]

nodes:
  igw01:
    id: 11
    config: [ extra-igw01.j2 ]
  igw02:
    id: 12
    config: [ extra-igw02.j2 ]

I would have expected that the extra config defined at node level would have been execute at the end of other configurations.

Instead, I have a slightly different behavior for igw01 and igw02:

PLAY [Deploy custom deployment templates] ****************************************************************************************************************************************************************************************

TASK [Run custom configuration deployment scripts] *******************************************************************************************************************************************************************************
skipping: [igw01] => (item=extra-igw02.j2)
skipping: [igw02] => (item=extra-igw01.j2)
included: /root/GIT_H/netlab/netsim/ansible/tasks/deploy-custom-config.yml for igw01 => (item=extra-igw01.j2)
included: /root/GIT_H/netlab/netsim/ansible/tasks/deploy-custom-config.yml for igw01, igw02 => (item=bgp.session)
included: /root/GIT_H/netlab/netsim/ansible/tasks/deploy-custom-config.yml for igw01, igw02 => (item=bgp.policy)
included: /root/GIT_H/netlab/netsim/ansible/tasks/deploy-custom-config.yml for igw02 => (item=extra-igw02.j2)

Will try to figure out what causes this "dis-order".

[ipspace]

So you'd like extra config to be very last? I can try to fix that, but it will be a bit messy. Right now, we append the plugin config to the extra config. Prepending wouldn't work unless we define plugin dependencies (which we could) as it would turn around the plugin order

In any case, the final order is computed somewhere close to Ansible output module to deal with the crappy way Ansible implements loops.

[ssasso]

Ok, I will deal with it, no problem.

And will deal with ansible loop odd-ity (which in any case has a different order for igw01 and igw02)

[ipspace]

Can't reproduce your behavior. This is the order in which the extra configs would be executed for me (from group_vars/custom_configs/topology.yml):

netlab_custom_config:
- extra-igw02.j2
- extra-igw01.j2
- bgp.session
- bgp.policy

Still, the extra config should be applied after plugin config, so I'll fix that.

[ipspace]

OK, I added the code that makes sure the plugin custom configs are execute before node custom configs. The plugin configs should execute before the node configs no matter what, the exact order of node config execution is unknown as I'm using sets and topological sort (where not everything is strictly comparable) and you know the sets are not ordered and topological sort can result in multiple "correct" orders.

[ipspace]

Would still love to see your group_vars/custom_configs/topology.yml (before pulling down the fix ;)

[ssasso]

(sorry for the delay)

This is the content of the file when I opened this issue:

# Ansible inventory created from ['/root/TOPOLOGIES/bugs/extra_disorder/topology.yml', 'package:topology-defaults.yml']
#

netlab_custom_config:
- extra-igw02.j2
- extra-igw01.j2
- bgp.session
- bgp.policy

So the order here is respecting what you said... but ansible for whatever reason decided to use a different one :/

[ssasso]

And this is the file with the latest dev:

# Ansible inventory created from ['/root/TOPOLOGIES/bugs/extra_disorder/topology.yml', 'package:topology-defaults.yml']
#

netlab_custom_config:
- bgp.session
- bgp.policy
- extra-igw02.j2
- extra-igw01.j2

[ipspace]

Thanks for the follow-up information. In any case, we fixed the "plugins before custom configs" dilemma.

As for "Ansible will do stuff in whatever order it likes", I'm no longer surprised. FWIW: what Ansible version are you using?

[ssasso]

root@hippo:~# ansible --version
ansible [core 2.13.13]
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0]
  jinja version = 3.1.4
  libyaml = True

[ipspace]

Thank you. I thought we fixed the reordering for good, particularly after I implemented the same tricks described in https://blog.ipspace.net/2022/03/ansible-reorder-list-values.html to custom configs.

On a more positive note, you're using a somewhat older release (corresponding to Ansible 6.x). I'm on 9.1 (Ansible core 2.16) and haven't experienced reordering (lately), so let's hope they got their **** together.