Trouble listing hosts in subgroups

[emollusion]

Hi!

I am having trouble listing hosts recursively trhough subgroups. It seems like it is not truly recursive but stops after a few levels. The structure for example could look like

• top:
• • middle:
• • • sub:
• • • • bottom:

Hosts exists in sub and bottom groups but not middle and top. I can list hosts from middle, sub and bottom groups but never top. If I list hosts from middle, I do not get bottom, if I list hosts from sub, I do get bottom.

Is there a way to modify this behaviour for either being truly recursive or at least being able to list one or two more levels?

Thank you!

[theasp]

Can you show me the yaml you are trying to use? This plugin does not allow defining groups in a tree like that, but the default Ansible YAML inventory plugin does.

[emollusion]

I can give you a simplified example from the inventory, please see below.

Calling group os will not give me any hosts. Calling group ubuntu or ubuntu_20 will return hosts. Calling group dns or dns_environment/dns_role will not return any hosts. Calling group dns_staging, dns_staging_internal, nscontrol, nschild will return hosts.

This also affects vars which are presented to each group. So calling a group to low will not bring along the vars set too high above. So if I call group dns_staging_internal, I will not get vars from dns.

As I have understood the plugin it should not put a cap on how a specific number of levels but maybe the entire functionality described here is out of scope?

---
groups:

# OS Level Groups
  os:
    include:
      - linux
  linux:
    include:
      - ubuntu
  ubuntu:
    include:
      - ubuntu_20

# Function Level Groups
  dns:
    include:
      - dns_environment
      - dns_role
  dns_environment:
    include:
      - dns_staging
  dns_role:
    include:
      - nscontrol
      - nschild
  dns_staging:
    include:
      - dns_staging_internal
      - dns_staging_external

# HOSTS
hosts:
  host_a:
    vars:
      ansible_host: 192.168.0.2
    groups:
      - ubuntu_20
      - dns_staging_internal
      - nscontrol
  host_b:
    vars:
      ansible_host: 192.168.0.3
    groups:
      - ubuntu_20
      - dns_staging_internal
      - nschild

[theasp]

Ah! What you want is not currently supported without adding extra lines to each set of includes. To accomplish this currently you would need to duplicate the includes for each group you are constructing:

groups:
  os:
    include:
      - linux
      - ubuntu
      - ubuntu_20
  linux:
    include:
      - ubuntu
      - ubuntu_20
  ubuntu:
    include:
      - ubuntu_20

Right now the groups are built in the sorted order of their names. To accomplish what you describe would require determining the correct order. The three methods I can think of for this, in order of preference, are:

1. Use topological sorting. The obvious choice for this would be the library toposort, however I don't want to add additional dependencies. I think this plugin should work out of the box with just the normal Ansible dependencies installed.
2. Each group could have a priority value that could be sorted on. As it turns out there is a special variable ansible_group_priority which is used to determine which group will take precedence when a variable is defined in multiple groups that a host is part of, which could be also used for this. As far as I can tell, that variable only works in the inventory file.
3. Provide an ordered list of groups (top level group_order item), then do any groups that aren't in that list.

I don't have much time to work on this in the short term, but I would gladly accept a PR that builds the groups in the correct order. Option 1 would be the most logically correct (assuming we don't add a dependency), but option 2 would match some of Ansible's existing variable precedence behaviour which might be less surprising.

[theasp]

Hi @emollusion, I just added toposort, can you let me know if this works as you expect?

[emollusion]

Hi @theasp No, it does not seem to be working after I have updated the yaml_groups.py file. But am I missing any dependencies? I do not get any warnings apart from ansible itself. Requesting group ubuntu does not return any hosts at all, requesting group ubuntu_20 returns hosts.

[theasp]

@emollusion, sorry for taking so long to get back to you... There are no additional dependencies, the toposort code is included in the module.

With the current version (https://github.com/theasp/ansible-inventory-yaml-groups/commit/3623ccca7baaea300c3103c097e27565747b8b22), your file from https://github.com/theasp/ansible-inventory-yaml-groups/issues/3#issuecomment-1252775061 seems to work for me?

xxx:~/projects/ansible-inventory-yaml-groups/test$ ansible -i hosts-emollusion.yml --list-hosts ubuntu
  hosts (2):
    host_a
    host_b
xxx:~/projects/ansible-inventory-yaml-groups/test$ ansible -i hosts-emollusion.yml --list-hosts os
  hosts (2):
    host_a
    host_b
xxx:~/projects/ansible-inventory-yaml-groups/test$ ansible -i hosts-emollusion.yml --list-hosts linux
  hosts (2):
    host_a
    host_b