search

redhat-cop / aap_configuration

A collection of roles to manage Ansible Controller and previously Ansible Tower
https://galaxy.ansible.com/infra/controller_configuration
GNU General Public License v3.0
268 stars 141 forks source link

Unable to use filetree_read role #885

Closed anderpups closed 2 months ago

anderpups commented 2 months ago

Summary

I posted this in the Matrix chat but it was requested that I open an issue.

I am unable to use the filetree_read role to set needed variables for the dispatch role after using the filetree_create role.

Issue Type

Ansible, Collection, Controller details

ansible --version
ansible [core 2.16.6]
  config file = /home/odtebja/Desktop/repos/dti_aap/ansible.cfg
  configured module search path = ['/home/odtebja/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/odtebja/Desktop/venvs/ansible/lib/python3.12/site-packages/ansible
  ansible collection location = /home/odtebja/Desktop/repos/dti_aap/collections
  executable location = /home/odtebja/Desktop/venvs/ansible/bin/ansible
  python version = 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] (/home/odtebja/Desktop/venvs/ansible/bin/python3)
  jinja version = 3.1.3
  libyaml = True

ansible-galaxy collection list
Collection                               Version
---------------------------------------- -------
ansible.controller                       4.5.8  
ansible.posix                            1.5.4  
community.crypto                         2.21.0 
community.general                        9.0.1  
infra.aap_utilities                      2.5.2  
infra.controller_configuration           2.9.0  
infra.eda_configuration                  1.1.0  
infra.ee_utilities                       3.2.0  
kubernetes.core                          5.0.0  

Controller version - 4.3.19

OS / ENVIRONMENT

RHEL8 for controller. Running playbook from Ubuntu 24.04 desktop

Desired Behavior

I am using the filetree_create role to produce a folder structure with our environment's settings. I thought I would be able to then use the filetree_read role to then read in the variables with the configs prior to using the dispatch role to actually push the config to a new install. It seems though that the filetree_read role requires a different folder structure (like this example). I feel like I am missing something simple.

Actual Behavior

Below is the directory structure created using the filetree_create role.

.
├── Admin
│   ├── current_organization.yaml
│   └── users
│       ├── bradley.anderson@delaware.gov.yaml
│       └── Patrick.Boyle@delaware.gov.yaml
├── Automation
│   ├── credentials
│   │   ├── 10_Automation - Shared Vault Passphrase.yaml
│   │   ├── 11_Automation - GitHub Source Control Credential.yaml
│   │   ├── 13_Automation - Public Ansible Galaxy.yaml
│   │   ├── 14_Automation - Ansible Windows User.yaml
│   │   ├── 7_Automation - Ansible User.yaml
│   │   └── 9_Automation - Vault Passphrase.yaml
│   ├── current_organization.yaml
│   ├── inventories
│   │   ├── Automation - AAP Lab
│   │   │   ├── 1_Automation - AAP Lab.yaml
│   │   │   └── current_inventory_sources.yaml
│   │   ├── Automation - GitHub Runners Lab
│   │   │   ├── 2_Automation - GitHub Runners Lab.yaml
│   │   │   └── current_inventory_sources.yaml
│   │   └── splunk-server
│   │       ├── 9_splunk-server.yaml
│   │       └── current_inventory_sources.yaml
│   ├── job_templates
│   │   ├── 10_Automation - AAP - Refresh AutomationHub Token.yaml
│   │   ├── 11_Automation - AAP - Daily Converge.yaml
│   │   ├── 13_Automation - GitHub Runners - Linux - Daily Converge.yaml
│   │   ├── 14_Automation - GitHub Runners - Windows - Daily Converge.yaml
│   │   ├── 16_Automation - AAP - Sync AutoHub Remote Registries.yaml
│   │   ├── 17_Automation - AAP - Take backup.yaml
│   │   ├── 78_Automation - Ping - Linux GitHub Runner.yaml
│   │   ├── 80_Automation - Ping - AAP.yaml
│   │   ├── 87_Automation - Ping Hosts.yaml
│   │   ├── 97_splunk server - lab.yaml
│   │   └── 9_Automation - AAP - Run Health Tests.yaml
│   ├── notification_templates
│   │   ├── Automation - Teams - Notifications - Error.yaml
│   │   ├── Automation - Teams - Notifications - Info.yaml
│   │   ├── Automation - Teams - Playground.yaml
│   │   └── Email Test.yaml
│   ├── projects
│   │   ├── 6_Automation - dti_ansible_automation_platform.yaml
│   │   ├── 77_Automation - dti_ansible_ping.yaml
│   │   ├── 7_Automation - dti_system_github_runners.yaml
│   │   └── 95_splunk-server.yaml
│   ├── teams
│   │   └── 1_Automation.yaml
│   ├── users
│   │   ├── bradley.anderson@delaware.gov.yaml
│   │   └── Patrick.Boyle@delaware.gov.yaml
│   └── workflow_job_templates
│       ├── 15_Automation - GitHub Runners - Daily Converge.yaml
│       └── 79_Automation - Workflow - Ping - Linux GitHub Runner.yaml
├── current_credential_types.yaml
├── current_execution_environments.yaml
├── current_instance_groups.yaml
├── current_settings.yaml
├── DCI
│   ├── credentials
│   │   ├── 12_DCI - GitHub Source Control.yaml
│   │   ├── 17_DCI - AAP PAT.yaml
│   │   ├── 18_DCI-Lab-vCenter.yaml
│   │   ├── 19_DCI - UCS.yaml
│   │   ├── 21_DCI - Vault.yaml
│   │   ├── 22_DCI - VMWare Host.yaml
│   │   ├── 24_DCI - GitHub Service Account.yaml
│   │   ├── 27_DCI - vRops.yaml
│   │   ├── 28_DCI - Unity.yaml
│   │   ├── 29_DCI - NetBox Token.yaml
│   │   ├── 30_DCI - MDS Lab Creds.yaml
│   │   ├── 31_DCI - MDS NXOS API.yaml
│   │   ├── 32_Lab ACI.yaml
│   │   ├── 33_DCI - Desktop.yaml
│   │   ├── 36_DCI - VMA Service Account.yaml
│   │   └── 8_DCI - Ansible User.yaml
│   ├── current_organization.yaml
│   ├── inventories
│   │   ├── ACI - Lab
│   │   │   ├── 8_ACI - Lab.yaml
│   │   │   └── current_inventory_sources.yaml
│   │   ├── DCI - Host Automation
│   │   │   ├── 3_DCI - Host Automation.yaml
│   │   │   └── current_inventory_sources.yaml
│   │   ├── DCI - localhost
│   │   │   ├── 5_DCI - localhost.yaml
│   │   │   └── current_hosts.yaml
│   │   ├── DCI - MDS Switches
│   │   │   └── 7_DCI - MDS Switches.yaml
│   │   ├── DCI - VMA
│   │   │   ├── 6_DCI - VMA.yaml
│   │   │   └── current_inventory_sources.yaml
│   │   └── DCI - VMWare UMDS
│   │       ├── 4_DCI - VMWare UMDS.yaml
│   │       └── current_inventory_sources.yaml
│   ├── job_templates
│   │   ├── 21_DCI - Converge UMDS.yaml
│   │   ├── 24_DCI - Update Reboot-Cluster Survey.yaml
│   │   ├── 25_DCI - Converge Infrastructure Automation Servers.yaml
│   │   ├── 26_DCI - Clone VM from Snapshot.yaml
│   │   ├── 27_DCI - VMware - Configure DRS.yaml
│   │   ├── 28_DCI - Converge vCenter and Hosts.yaml
│   │   ├── 29_DCI - Reboot VMware Cluster.yaml
│   │   ├── 33_DCI - SNOW-Q.yaml
│   │   ├── 36_DCI - Add Automated Hosts to Cluster.yaml
│   │   ├── 37_DCI - SNOW-CMDB.yaml
│   │   ├── 38_DCI - Host Automation - Part 1.yaml
│   │   ├── 40_DCI - Host Automation - Part 2.yaml
│   │   ├── 41_DCI - Host Automation - Part 3.yaml
│   │   ├── 42_DCI - UCS Vlan Config.yaml
│   │   ├── 44_DCI - Billing.yaml
│   │   ├── 45_DCI - Converge VMA Server.yaml
│   │   ├── 46_DCI - Daily Maintenance.yaml
│   │   ├── 47_DCI - Farm Report.yaml
│   │   ├── 55_DCI - Daily Snapshot Maintenance.yaml
│   │   ├── 56_DCI - vCheck Report.yaml
│   │   ├── 59_DCI - Hourly Maintenance.yaml
│   │   ├── 68_DCI - MDS - AutoZoning - Fabric A.yaml
│   │   ├── 69_DCI - MDS - AutoZoning - Fabric B.yaml
│   │   ├── 71_DCI - MDS - Backup Config.yaml
│   │   ├── 74_ACI - Update ACI.yaml
│   │   ├── 75_DCI - ACI - Rollback 1.yaml
│   │   ├── 82_DCI - Converge Ubuntu Desktop.yaml
│   │   ├── 86_DCI - Manage Github Repos.yaml
│   │   └── 93_DCI - vCenter DSP.yaml
│   ├── labels
│   │   ├── 10_report.yaml
│   │   ├── 11_snow.yaml
│   │   ├── 13_powershell.yaml
│   │   ├── 14_vmware.yaml
│   │   ├── 15_ucs.yaml
│   │   ├── 7_lab.yaml
│   │   ├── 8_vma.yaml
│   │   └── 9_maintenance.yaml
│   ├── notification_templates
│   │   ├── DCI - Email.yaml
│   │   ├── Email-PD.yaml
│   │   ├── Mattermost - Tower - DCI - Verbose.yaml
│   │   ├── Mattermost - Tower - DCI.yaml
│   │   ├── Teams-DCI-Updates.yaml
│   │   └── Teams - DCI - Verbose.yaml
│   ├── projects
│   │   ├── 18_vmware-umds.yaml
│   │   ├── 19_infrastructure-automation.yaml
│   │   ├── 23_vmware-mgmt.yaml
│   │   ├── 30_vma.yaml
│   │   ├── 31_SNOW-Q.yaml
│   │   ├── 32_SNOW-CMDB.yaml
│   │   ├── 67_Ansible-Zoning.yaml
│   │   ├── 72_aci.yaml
│   │   ├── 81_DCI-Ubuntu-Desktop.yaml
│   │   ├── 84_dti_dci_team.yaml
│   │   ├── 88_vmware-definitions.yaml
│   │   └── 91_vcenter-profile.yaml
│   ├── teams
│   │   └── 2_DCI.yaml
│   └── workflow_job_templates
│       ├── 66_DCI - Host Automation.yaml
│       ├── 70_DCI - MDS - AutoZoning Workflow.yaml
│       ├── 76_ACI - Workflow - Update ACI.yaml
│       └── 92_DCI - Host Automation - Post Processing.yaml
├── NetSec
│   ├── credentials
│   │   └── 37_NetSec - vault.yaml
│   ├── current_organization.yaml
│   └── teams
│       └── 3_NetSec.yaml
├── ORGANIZATIONLESS
│   ├── credentials
│   │   ├── 1_Default Execution Environment Registry Credential.yaml
│   │   ├── 20_DCI-VMware-Host.yaml
│   │   ├── 25_Pat D UCS.yaml
│   │   ├── 26_DCI - Vmware Autodeploy.yaml
│   │   ├── 2_Automation Hub Validated Repository.yaml
│   │   ├── 35_ghcr.io.yaml
│   │   ├── 3_Automation Hub Published Repository.yaml
│   │   ├── 4_Automation Hub RH Certified Repository.yaml
│   │   ├── 5_Automation Hub Community Repository.yaml
│   │   └── 6_Automation Hub Container Registry.yaml
│   └── users
│       ├── aap_jobs_automation_automation_svc.yaml
│       ├── aap_jobs_automation_dci_svc.yaml
│       ├── aap_jobs_automation_netsec_svc.yaml
│       ├── aap.test.netsec@delaware.gov.yaml
│       ├── aap.test.user@delaware.gov.yaml
│       ├── admin.yaml
│       ├── ansible_automation_platform_tests.yaml
│       ├── Arthur.Powell@delaware.gov.yaml
│       ├── banderson.adm@delaware.gov.yaml
│       ├── github_automation_sandbox@delaware.gov.yaml
│       ├── Justin.Keplinger@delaware.gov.yaml
│       ├── Patrick.Dougherty@delaware.gov.yaml
│       ├── paul.knight@delaware.gov.yaml
│       ├── pboyle.adm@delaware.gov.yaml
│       └── Philip.Cheatham@delaware.gov.yaml
├── schedules
│   ├── 10_Automation - GitHub Runners - Daily Converge - Every Day.yaml
│   ├── 11_Automation - AAP - Sync AutomationHub Remote EE Registries - Sunday@2200.yaml
│   ├── 12_Automation - AAP - Take Backup - Daily.yaml
│   ├── 13_Daily 6am.yaml
│   ├── 14_Everyday 5AM.yaml
│   ├── 15_Daily.yaml
│   ├── 16_every day at 1900.yaml
│   ├── 17_DRS Rules Daily.yaml
│   ├── 18_VMA - Daily Maintenance.yaml
│   ├── 19_VMA - Daily Snapshot Maintenance.yaml
│   ├── 1_Cleanup Job Schedule.yaml
│   ├── 20_VMA - Daily Converge Server.yaml
│   ├── 21_VMA - Hourly Maintenance.yaml
│   ├── 22_VMA - Daily vCheck Report.yaml
│   ├── 23_every 4 hrs.yaml
│   ├── 24_Daily.yaml
│   ├── 25_manage repos daily.yaml
│   ├── 26_DCI - Run Health Tests.yaml
│   ├── 27_DCI - Monday Morning.yaml
│   ├── 28_splunk serer - daivly.yaml
│   ├── 2_Cleanup Activity Schedule.yaml
│   ├── 4_Cleanup Expired Sessions.yaml
│   ├── 5_Cleanup Expired OAuth 2 Tokens.yaml
│   ├── 6_Automation - Sync AutomationHub Token - Run every 14 days.yaml
│   ├── 7_Automation - AAP - Daily Converge - Once a Day.yaml
│   └── 9_Automation - AAP - Run Tests.yaml
├── team_roles
│   ├── current_roles_Automation.yaml
│   ├── current_roles_DCI.yaml
│   └── current_roles_NetSec.yaml
├── TestOrg
│   ├── current_organization.yaml
│   ├── teams
│   │   └── 4_TestTeam.yaml
│   └── users
│       └── Patrick.Boyle@delaware.gov.yaml
└── user_roles
    ├── current_roles_aap_jobs_automation_automation_svc.yaml
    ├── current_roles_aap_jobs_automation_dci_svc.yaml
    ├── current_roles_aap_jobs_automation_netsec_svc.yaml
    ├── current_roles_aap.test.netsec@delaware.gov.yaml
    ├── current_roles_aap.test.user@delaware.gov.yaml
    ├── current_roles_ansible_automation_platform_tests.yaml
    ├── current_roles_github_automation_sandbox@delaware.gov.yaml
    ├── current_roles_Justin.Keplinger@delaware.gov.yaml
    ├── current_roles_Patrick.Dougherty@delaware.gov.yaml
    ├── current_roles_paul.knight@delaware.gov.yaml
    └── current_roles_Philip.Cheatham@delaware.gov.yaml

Below is the playbook I am using:

---
- name: Playbook to configure ansible controller post installation
  hosts: aap_orchestrator_node
  vars:
    controller_hostname: "{{ aap_controller_hostname }}"
    controller_password: "{{ aap_admin_password }}"
    controller_username: "{{ controller_user | default('admin') }}"
    controller_validate_certs: false
  pre_tasks:
    - name: Setup authentication
      block:
        - name: "Get the Authentication Token for the future requests"
          ansible.builtin.uri:
            url: "https://{{ controller_hostname }}/api/v2/tokens/"
            user: "{{ controller_username }}"
            password: "{{ controller_password }}"
            method: POST
            force_basic_auth: true
            validate_certs: "{{ controller_validate_certs }}"
            status_code: 201
          register: authtoken_res

        - name: "Set the oauth token to be used since now"
          ansible.builtin.set_fact:
            controller_oauthtoken: "{{ authtoken_res.json.token }}"
            controller_oauthtoken_url: "{{ authtoken_res.json.url }}"
            controller_configuration_credentials_secure_logging: false
            orgs: Automation
            dir_orgs_vars: ./import/moleucle
      no_log: "{{ controller_configuration_filetree_create_secure_logging | default('false') }}"
      when: controller_oauthtoken is not defined

  roles:
    - infra.controller_configuration.filetree_read
    - infra.controller_configuration.dispatch

  post_tasks:
    - name: "Delete the Authentication Token used"
      ansible.builtin.uri:
        url: "https://{{ controller_hostname }}{{ controller_oauthtoken_url }}"
        user: "{{ controller_username }}"
        password: "{{ controller_password }}"
        method: DELETE
        force_basic_auth: true
        validate_certs: "{{ controller_validate_certs }}"
        status_code: 204
      when: controller_oauthtoken_url is defined

When I run the playbook though, the filetree_read role looks for directories such as:

The entire console output is here. You can see where it just skips everything since the directory structure is not the same.

Thank you in advance for any direction or help you can provide.

ivarmu commented 2 months ago

You can override these variables in your playbook specifying the root directory of the filetree_create output:

https://github.com/redhat-cop/controller_configuration/blob/devel/roles%2Ffiletree_read%2Fdefaults%2Fmain.yml#L33

After this, filetree_read will work as you expect.

anderpups commented 2 months ago

@ivarmu - Thank you for your response. I met with the team during office hours and they pointed me in the right direction.