aristanetworks / avd

Arista Validated Designs
https://avd.arista.com
Apache License 2.0
290 stars 209 forks source link

Allow hosts to be members of many groups when deploying to CVP #4633

Open gadgieOps opened 2 days ago

gadgieOps commented 2 days ago

Enhancement summary

We are unable to implement our internal best practice inventory format when using the eos_config_deploy_cvp role. We implement ansible in many areas of our business and have standardised on a format that ensures our inventories are simple, clean and scalable. We are able to continue with the current behaviour however it would be highly helpful to us if this behaviour wasn't present. I suspect this is the intended behaviour of the role and not a bug however other roles operate correctly such as eos_designs and eos_cli_config_gen.

We integrate AVD with other automations and for us to scale our work, we would like to be able to keep our code across all solutions as standardised as possible. Our goals are to make network automation as approachable as possible for engineers who have previously had little exposure in our industry.

It is unlikely I'll be able to afford the time (Given we have workarounds) to study the role in enough depth to raise a PR with suggested changes however I will keep it in mind. If this causes wider issues moving forwards.

Which component of AVD is impacted

eos_config_deploy_cvp

Use case example

Here is a snippet from our desired AVD inventory (M&E Red/Blue network and simplified for brevity):

---
all:
  children:
    PROD:
      children:
        TESTLAB-01:
          children:
            MLAN:
              hosts:
                MLAN-SP-01:
                  ansible_host: mlan-sp-01.ourdomain.com
                MLAN-SP-02:
                  ansible_host: mlan-sp-02.ourdomain.com
                MLAN-VID-LF-01-R:
                  ansible_host: mlan-vid-lf-01-r.ourdomain.com
                MLAN-VID-LF-01-B:
                  ansible_host: mlan-vid-lf-01-b.ourdomain.com
                MLAN-VID-LF-02-R:
                  ansible_host: mlan-vid-lf-02-r.ourdomain.com
                MLAN-VID-LF-02-B:
                  ansible_host: mlan-vid-lf-02-b.ourdomain.com
                MLAN-AUD-LF-01-B:
                  ansible_host: mlan-aud-lf-01-b.ourdomain.com
                MLAN-AUD-LF-01-R:
                  ansible_host: mlan-aud-lf-01-r.ourdomain.com
              children:
                MLAN-RED:
                  hosts:
                    MLAN-SP-01:
                    MLAN-VID-LF-01-R:
                    MLAN-VID-LF-02-R:
                    MLAN-AUD-LF-01-R:
                MLAN-BLUE:
                  hosts:
                    MLAN-SP-02:
                    MLAN-VID-LF-01-B:
                    MLAN-VID-LF-02-B:
                    MLAN-AUD-LF-01-B:
                MLAN-SP:
                  hosts:
                    MLAN-SP-01:
                    MLAN-SP-02:
                MLAN-LF:
                  hosts:
                    MLAN-VID-LF-01-R:
                    MLAN-VID-LF-01-B:
                    MLAN-VID-LF-02-R:
                    MLAN-VID-LF-02-B:
                    MLAN-AUD-LF-01-R:
                    MLAN-AUD-LF-01-B:
            CVX:
              hosts:
                CVX-01-R:
                  ansible_host: cvx-01-r.ourdomain.com
                CVX-01-B:
                  ansible_host: cvx-01-b.ourdomain.com
                CVX-02-R:
                  ansible_host: cvx-02-r.ourdomain.com
                CVX-02-B:
                  ansible_host: cvx-02-b.ourdomain.com
                CVX-03-R:
                  ansible_host: cvx-03-r.ourdomain.com
                CVX-03-B:
                  ansible_host: cvx-03-b.ourdomain.com
              children:
                CVX-RED:
                  CVX-01-R:
                  CVX-02-R:
                  CVX-03-R:
                CVX-BLUE:
                  CVX-01-B:
                  CVX-02-B:
                  CVX-03-B:
            CLAN:
              hosts:
                CLAN-SP-01:
                  ansible_host: clan-sp-01.ourdomain.com
                CLAN-SP-02:
                  ansible_host: clan-sp-02.ourdomain.com
                CLAN-LF-01:
                  ansible_host: clan-lf-01.ourdomain.com
                CLAN-LF-02:
                  ansible_host: clan-lf-02.ourdomain.com
                CLAN-LF-03:
                  ansible_host: clan-lf-03.ourdomain.com
                CLAN-LF-04:
                  ansible_host: clan-lf-04.ourdomain.com
                CLAN-LF-05:
                  ansible_host: clan-lf-05.ourdomain.com
              children:
                CLAN-SP:
                  hosts:
                    CLAN-SP-01:
                    CLAN-SP-02:
                CLAN-LF:
                  hosts:
                    CLAN-LF-01:
                    CLAN-LF-02:
                    CLAN-LF-03:
                    CLAN-LF-04:
                    CLAN-LF-05:
                CLAN-LF-01-MLAG:
                  hosts:
                    CLAN-LF-01:
                    CLAN-LF-02:
                CLAN-LF-04-MLAG:
                  hosts:
                    CLAN-LF-04:
                    CLAN-LF-05:

We organise inventories like this as it allows host ansible_host definition in a single part of the YAML structure and allows us lots of flexibility using ansible's group_var directory structure.

When running an inventory in this format, eos configs are generated correctly however when deploying to CVP, the key parentContainerName is duplicated per host in the CVP structured config. For example:

---
cvp_devices:

  - fqdn: MLAN-AUD-LF-01-B
    parentContainerName: MLAN
    parentContainerName: MLAN-BLUE
    parentContainerName: MLAN-LF
    configlets:
      - AVD-RALAB_MLAN-AUD-LF-01-B

Ansible error:

TASK [arista.avd.eos_config_deploy_cvp : Load CVP device information for cvp-01] *******************************************************************************************************************************************************************************************
fatal: [cvp-01]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "We were unable to read either as JSON nor YAML, these are the errors we got from each:\nJSON: Expecting value: line 1 column 1 (char 0)\n\nSyntax Error while loading YAML.\n  While constructing a mapping from /Users/dave/Projects/RALAB-AVD/configs/intended_structured_configs/cvp_structured_configs/cvp-01.yml, line 55, column 5, found a duplicate dict key (parentContainerName). Using last defined value only.\n\nThe error appears to be in '/Users/dave/Projects/RALAB-AVD/configs/intended_structured_configs/cvp_structured_configs/cvp-01.yml': line 55, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n      - AVD_CVX-03-R\n  - fqdn: MLAN-AUD-LF-01-B\n    ^ here\n"}

Describe the solution you would like

We would like eos_config_deploy_cvp to behave the same as eos_designs and eos_cli_config_gen and allow for hosts to be members of multiple groups. Furthermore, it would be great to decouple the anisble inventory structure from the container structure in CVP. This would allow more freedom to integrate with other automations and existing practices without compromising the operational structure in CVP.

Describe alternatives you have considered

Workarounds:

  1. Use the inventory format as documented (even though isn't a great fit for us)
  2. Add another group dedicated to hosts only and run eos_config_deploy_cvp multiple times with different the container_root set to the direct parent

Workaround 2:

---
- name: Push to CVP
  hosts: CVP
  gather_facts: false
  connection: local
  tasks:
      # Provision MLAN
    - name: Run CVP provisioning
      ansible.builtin.import_role:
        name: arista.avd.eos_config_deploy_cvp
      vars:
        container_root: MLAN-SWITCHES
        configlets_prefix: "AVD-MLAN"
        execute_tasks: false
        state: present
      tags: build, provision, mlan

      # Provision CLAN
    - name: Run CVP provisioning
      ansible.builtin.import_role:
        name: arista.avd.eos_config_deploy_cvp
      vars:
        container_root: CLAN-SWITCHES
        configlets_prefix: "AVD-CLAN"
        execute_tasks: false
        state: present
      tags: build, provision, clan

      # Provision CVX
    - name: Run CVP provisioning
      ansible.builtin.import_role:
        name: arista.avd.eos_config_deploy_cvp
      vars:
        container_root: CVX-INSTANCES
        configlets_prefix: "AVD-CVX"
        execute_tasks: false
        state: present
      tags: build, provision, cvx

Issue with workaround 2:

Additional context

No response

Contributing Guide

ClausHolbechArista commented 2 days ago

Thank you for submitting this issue with elaborate background and details.

The role eos_config_deploy_cvp needs a strict tree to create the container hierarchy. This tree is inferred from the Ansible inventory starting at the group set as root container. It is not possible to know which of the groups is the "correct one" from the Ansible inventory, if there are multiple parallel trees.

We could indeed decouple the container hierarchy from the inventory, and allow the user to pass in the hierarchy. This is already on the roadmap for our new role cv_deploy, which pushes configs to the new Static Config Studio on CloudVision.

We do not plan to change the behavior of eos_config_deploy_cvp, since the role will be replaced with cv_deploy. eos_config_deploy_cvp is tied to the older provisioning backend on CloudVision.

All that said, I am a bit confused about the output you see. I would expect an error to be raised by the module inventory_to_containers instead of producing duplicate output keys. Which version of AVD are you using?

gadgieOps commented 2 days ago

Thank you for the response. Glad to hear decoupling the inventory from the container hierarchy is on the roadmap for cv_deploy.

AVD 4.10.2.