kurokobo / awx-on-k3s

An example implementation of AWX on single node K3s using AWX Operator, with easy-to-use simplified configuration with ownership of data and passwords.
MIT License
513 stars 142 forks source link

How to use dynamic inventory from Netbox into AWX UI? #373

Closed vietsoulz closed 1 day ago

vietsoulz commented 2 weeks ago

k3s version v1.29.4+k3s1 (94e29e2e) go version go1.21.9

OS: Red Hat Enterprise Linux 8.10 (Ootpa) - Linux 4.18.0-553.el8_10.x86_64 Kubernetes/K3s: Client Version: v1.29.4+k3s1 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.29.4+k3s1 AWX Operator: 2.17.0

Netbox version: v3.7.3 running on RHEL8 on separate VM

Description

Hello @kurokobo, thanks for the use case 1 to mount local disk into AWX. I am looking to understand how to properly (step by step) to integrate Netbox devices (using netbox.netbox.nb_inventory plugin) to use as dynamic inventory in AWX.

I've taken Ansible's guide to create an inventory sourcing from a project with this yml code, but kept getting these messages/warnings. My project syncs from gitlab and I have executeable permission on all my files and they show up on the 'inventory file' drop-down menu.

This script worked great using the ansible-inventory command.

plugin: netbox.netbox.nb_inventory api_endpoint: "netbox_url" token: 'netbox_token' validate_certs: False config_context: False group_by:

AWX's sync output for the Inventories: ! host_list declined parsing /runner/project/netbox_inventory.yml as it did not pass its verify_file() method toml declined parsing /runner/project/netbox_inventory.yml as it did not pass its verify_file() method [WARNING]: Failed to parse /runner/project/netbox_inventory.yml with script plugin: problem running /runner/project/netbox_inventory.yml --list ([Errno 8] Exec format error: '/runner/project/netbox_inventory.yml') File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/script.py", line 150, in parse raise AnsibleParserError(to_native(e)) [WARNING]: Failed to parse /runner/project/netbox_inventory.yml with auto plugin: inventory config '/runner/project/netbox_inventory.yml' specifies unknown plugin 'netbox.netbox.nb_inventory' File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/auto.py", line 53, in parse raise AnsibleParserError("inventory config '{0}' specifies unknown plugin '{1}'".format(path, plugin_name)) [WARNING]: Failed to parse /runner/project/netbox_inventory.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/yaml.py", line 114, in parse raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory') [WARNING]: Failed to parse /runner/project/netbox_inventory.yml with ini plugin: Invalid host pattern '---' supplied, '---' is normally a sign this is a YAML file. File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/ini.py", line 137, in parse raise AnsibleParserError(e) [WARNING]: Unable to parse /runner/project/netbox_inventory.yml as an inventory source ERROR! No inventory was parsed, please check your configuration and options.

pip3 list output: Package Version


ansible-core 2.17.0 ansible-pylibssh 1.2.0.post4 cachetools 5.3.3 certifi 2024.6.2 cffi 1.16.0 chardet 5.2.0 charset-normalizer 3.3.2 colorama 0.4.6 cryptography 42.0.7 distlib 0.3.8 filelock 3.14.0 idna 3.7 Jinja2 3.1.4 MarkupSafe 2.1.5 netbox-api 0.4.3 packaging 23.2 pip 23.2.1 platformdirs 4.2.2 pluggy 1.5.0 pycparser 2.22 pynetbox 7.3.3 pyproject-api 1.6.1 pytz 2024.1 PyYAML 6.0.1 requests 2.32.3 resolvelib 1.0.1 setuptools 68.2.2 tabulate 0.9.0 tox 4.15.0 urllib3 2.2.1 virtualenv 20.26.2

Installed ansible-galaxy collections inside the awx_task/web pods: /var/lib/awx/projects/cisco/collections/ansible_collections bash-5.1$ total 0 drwxr-xr-x 4 root root 36 Jun 4 17:36 ansible drwxr-xr-x 2 root root 24 Jun 4 17:05 ansible.netcommon-6.1.3.info drwxr-xr-x 2 root root 24 Jun 4 17:36 ansible.utils-4.1.0.info drwxr-xr-x 3 root root 17 Jun 4 16:53 arista drwxr-xr-x 2 root root 24 Jun 4 16:53 arista.eos-9.0.0.info drwxr-xr-x 9 root root 101 Jun 4 16:52 cisco drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.asa-5.0.1.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.ios-8.0.0.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.iosxr-9.0.0.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.ise-2.9.1.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.nxos-8.1.0.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.sdwan-0.1.0.info drwxr-xr-x 2 root root 24 Jun 4 16:52 cisco.sdwan_deployment-0.1.1.info drwxr-xr-x 6 root root 65 Jun 12 19:34 community drwxr-xr-x 2 root root 24 Jun 4 16:52 community.general-7.0.0.info drwxr-xr-x 2 root root 24 Jun 12 19:34 community.general-9.0.1.info drwxr-xr-x 2 root root 24 Jun 4 16:53 community.grafana-1.8.0.info drwxr-xr-x 2 root root 24 Jun 4 16:53 community.network-5.0.2.info drwxr-xr-x 2 root root 24 Jun 4 16:53 community.zabbix-2.4.0.info drwxr-xr-x 3 root root 18 Jun 4 16:53 kubernetes drwxr-xr-x 2 root root 24 Jun 4 16:53 kubernetes.core-4.0.0.info drwxr-xr-x 3 root root 20 Jun 13 18:44 netbox drwxr-xr-x 2 root root 24 Jun 13 18:44 netbox.netbox-3.19.1.info bash-5.1$

Any help would be greatly appreciated. Thanks again!

Screenshot 2024-06-14 at 11 20 28 AM

kurokobo commented 2 weeks ago

@vietsoulz Hi, this is the root cause I think:

unknown plugin 'netbox.netbox.nb_inventory'

Please ensure your EE that used for inventory sync contains netbox.netbox collection. Inventory sync is invoked in ephemeral job pod as well as standard job, so in your screen, the EE that contains netbox collection should be specified here:

image

vietsoulz commented 2 weeks ago

@kurokobo Thank you. Do you have a specific guide that I can create the EE to work within your AWX deployment? I see a lot of guides online but not sure if it has to be specific with the AWX instance from your guide.

kurokobo commented 2 weeks ago

https://github.com/kurokobo/awx-on-k3s/blob/main/builder/README.md

kurokobo commented 2 weeks ago

I recommend you to try Use Ansible collections without custom EE section first.

vietsoulz commented 2 weeks ago

These are the two built-in EEs that came with your deployment. Is there a way to modify it and update the Ansible collections?

Screenshot 2024-06-15 at 8 11 59 PM

I will try to add the netbox.netbox source under the Use Ansible Collection without custom EE to see if that resolves my issues. I did have that file populated with all of my Cisco modules and it was working so far.

[root@pgsscvntwkawx02 collections]# cat requirements.yml

collections:

kurokobo commented 2 weeks ago

These are the two built-in EEs that came with your deployment. Is there a way to modify it and update the Ansible collections?

The built-in EEs are came from here: https://github.com/ansible/awx-ee If you want to modify them, modify the source code and built it.

vietsoulz commented 2 weeks ago

I struggled a bit to get all the modules I need and able to create a custom EE on my local AWX host. How do I use this custom EE in AWX UI? Screenshot 2024-06-18 at 10 58 55 AM

Screenshot 2024-06-18 at 11 02 06 AM

I got this message when attempting to run the Inventories job from the Custom EE that I created:

Receptor detail: Error creating pod: container failed to start, ImagePullBackOff

kurokobo commented 2 weeks ago

@vietsoulz I already provide the guide to use custom EE with your AWX with or without container registry. Please see my guide again, its Use EE in AWX section: https://github.com/kurokobo/awx-on-k3s/tree/main/builder#use-ee-in-awx

github-actions[bot] commented 5 days ago

This issue is stale because it has been open 10 days with no activity. Remove stale label or comment or this will be closed in 4 days.

github-actions[bot] commented 1 day ago

This issue was closed because it has been open 2 weeks with no activity.