Netbox Inventory Plugin for AWX/Tower

[loganbest]

ISSUE TYPE

• Feature Idea

SUMMARY

As of Ansible 2.8, Netbox has had a native inventory plugin. Including that as an available inventory source in AWX/Tower would be a welcome addition in the automation world as Netbox's popularity increases.

[loganbest]

@fragmentedpacket

[petrdanecek]

what is the workaround for now? I mean I tested the ansible netbox plugin and it works like a charm. Do I need to use some kind of custom script in the AWX until it is implemented into AWX?

[ryanpetrello]

@petrdanecek,

AWX supports custom inventory scripts. This is one way to accomplish this for now: https://docs.ansible.com/ansible-tower/latest/html/administration/custom_inventory_script.html

[ryanpetrello]

@AlanCoding,

Is there any way to do this with the new plugin architecture? i.e., have some custom source that - instead of a script - is literally just "use plugin X from Ansible"?

[wenottingham]

The short-term way to use an inventory plugin is to place the configuration file for the plugin in source control in an AWX project, and point the inventory source at that.

[loganbest]

The main blocker to bringing it natively is that the inventory plugin structure is different than with Ansible. The same plugin can be used but just needs to be modified slightly to adhere to the different format. Boggles my mind why all native ansible inventory plugins aren't automatically usable in AWX really.

[AlanCoding]

Replacing this issue with https://github.com/ansible/awx/issues/4306

Basically, allow users to enter the plugin configuration without having to go through source control. Let me know if I have misunderstood anything.

[emil-palm]

@wenottingham

The short-term way to use an inventory plugin is to place the configuration file for the plugin in source control in an AWX project, and point the inventory source at that.

Can you provide some more insight into this solution, i tried to put my yaml config file in my repo and point the inventory to that file but that just failed with

  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/var/lib/awx/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-inventory
  python version = 2.7.5 (default, Jun 20 2019, 20:27:34) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /var/lib/awx/projects/_8__test/ansible/hosts/hosts as it did not pass it's verify_file() method
script declined parsing /var/lib/awx/projects/_8__test/ansible/hosts/hosts as it did not pass it's verify_file() method
auto declined parsing /var/lib/awx/projects/_8__test/ansible/hosts/hosts as it did not pass it's verify_file() method
toml declined parsing /var/lib/awx/projects/_8__test/ansible/hosts/hosts as it did not pass it's verify_file() method
 [WARNING]:  * Failed to parse
/var/lib/awx/projects/_8__test/ansible/hosts/hosts with yaml plugin: Plugin
configuration YAML file, not YAML inventory
  File "/usr/lib/python2.7/site-packages/ansible/inventory/manager.py", line 267, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/inventory/yaml.py", line 112, in parse
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
 [WARNING]:  * Failed to parse
/var/lib/awx/projects/_8__test/ansible/hosts/hosts with ini plugin: Invalid
host pattern 'plugin:' supplied, ending in ':' is not allowed, this character
is reserved to provide a port.
  File "/usr/lib/python2.7/site-packages/ansible/inventory/manager.py", line 267, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/inventory/ini.py", line 138, in parse
    raise AnsibleParserError(e)
 [WARNING]: Unable to parse /var/lib/awx/projects/_8__test/ansible/hosts/hosts
as an inventory source
ERROR! No inventory was parsed, please check your configuration and options.```

[AlanCoding]

auto declined parsing /var/lib/awx/projects/_8__test/ansible/hosts/hosts as it did not pass it's verify_file() method

The verify_file method mentioned:

https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/inventory/auto.py#L34-L37

Sounds like this error is telling me that you gave it a file that did not end in .yml/.yaml, as the netbox inventory plugin documents

netbox_inventory.yml file in YAML format

[sakbhav]

I wrote a small inventory script to use netbox as inventory:

https://github.com/sakbhav/netbox-awx

[jdrew82]

I wrote a small inventory script to use netbox as inventory:

https://github.com/sakbhav/netbox-awx

I don't see this script working as it appears you never actually populate the devices list. I see you also have a hosts_list variable that I'm assuming is a duplicate variable? I updated script to use that and it appears to be working now outside of the tenants portion crashing it. Works when commented out though. I do see that facts for the hosts don't appear to be populated though. Good start for the script but needs some tweaking.

[sakbhav]

Thanks for reporting @jdrew82 ! It was my bad. I weeded out some of my organisation specific code to filter out hosts based on specific tags. I think this could be useful for everyone, so I have added it back. I have fixed the code and added some additional checks for null values. Please let me know if its working for you now or not.

[jdrew82]

Much better now! I'm able to have everything uncommented and it appears to populate more info than before. I've started tweaking it a bit to pull more information from Netbox for Ansible but it's a good start. Very much appreciated!

[sakbhav]

Glad to know it worked for you. Please feel free to raise a Pull Request if you find something worth adding to this script for general usecase.

[zedr]

I've written a new example script from scratch, if anyone needs it: https://gist.github.com/zedr/6979ab2fc49fe13e752a9896d6195c4d

[AndrewSav]

@zedr what's the plan with regards to this when custom scripts are removed in the next version of awx?

[AlanCoding]

The forward-looking way to use the netbox inventory would be to use https://galaxy.ansible.com/netbox/netbox

You can create a project with your inventory file and put a file collections/requirements.yml that lists netbox.netbox in a way that ansible-galaxy CLI can understand. That should work now.

With execution environments, #8030, we could talk about an additional feature to allow direct input of the inventory config, where the collection is sourced from the container image.

[zedr]

I've refashioned the Inventory Script into a proper Ansible Inventory Plugin: https://github.com/zedr/simple_netbox_inventory_plugin

[joubbi]

I found this bug report when trying to get Tower use NetBox as an inventory. I took @zedr (Thank you!) code example from above and made a few improvements to it. It can be found here: https://github.com/joubbi/towerbox Just in case someone else comes here for the same purpose.

[ogowen45]

I am having issues getting Netbox to populate the inventory of AWX. I have tried adding collections/requirements.yml listing netbox.netbox and netbox_inventory.yml file and creating a project with the inventory file like @AlanCoding suggested but only getting the same errors as mrevilme was.

ansible-inventory [core 2.11.6.post0] config file = None configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python3.8/site-packages/ansible ansible collection location = /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections:/usr/share/automation-controller/collections executable location = /usr/local/bin/ansible-inventory python version = 3.8.6 (default, Jan 29 2021, 17:38:16) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)] jinja version = 2.10.3 libyaml = True No config file found; using defaults host_list declined parsing /bin/[ as it did not pass its verify_file() method auto declined parsing /bin/[ as it did not pass its verify_file() method Parsed /bin/[ inventory source with ini plugin host_list declined parsing /bin/addr2line as it did not pass its verify_file() method auto declined parsing /bin/addr2line as it did not pass its verify_file() method toml declined parsing /bin/addr2line as it did not pass its verify_file() method [WARNING]: * Failed to parse /bin/addr2line with script plugin: Inventory script (/bin/addr2line) had an execution error: /bin/addr2line: unrecognized option '--list' Usage: /bin/addr2line [option(s)] [addr(s)] Convert addresses into line number/file name pairs. If no addresses are specified on the command

From cli I can run ansible-inventory -i inventory.yml and pull all the devices in fine, with the same config in that inventory file as is in what AWX is using. @joubbi I attempted to run the script you provided but ended up with similar errors

Am I missing something?

[zedr]

@ogowen45 what is addr2line? It's being called as the inventory script, but it cannot handle the argument --list.

[ogowen45]

It had to do with there being a space in the file location in the inventory source, as well as there was no ansible.cfg file in the repo.

After, I made an ansible.cfg file for repository that is as follows Now getting error

[AlanCoding]

I think you basically need to create an execution environment with the collection and its dependencies (like jsonschema). This is the problem they were intended to solve.