Add HTTP Inventory Source

[gmtime]

ISSUE TYPE

• Since Custom Inventory Scripts are deprecated, it would be nice to be able to fetch a JSON inventory file from a remote URL

SUMMARY

• Custom Inventory Scripts are deprecated. Adding an inventory file into SCM is an option, but it doesn't quite work in fast changing environments (in fact it works, but it's painful)

[nifr]

Inventory scripts themselves are not deprecated. You can still use inventory scripts stored inside SCM projects as "Sources" in an inventory.

Only the option "Custom Inventory Script" that stores the script content in awx's database - making it editable via REST API / in the UI - was deprecated. It makes sense to put your inventory scripts under version control in general.

1. create a SCM repository
2. add your inventory script
3. add the SCM repository to awx as a "Project"
4. create a new "Inventory" in awx
5. open the inventory, go to "Sources" and click "Add"
6. select "Sourced from a Project" as "Source"
7. select the SCM repository project as "Project"
8. select your inventory script (or a directory containing multiple files) under "Inventory File"

Done.

---

Example Shell Script:

In order to query a JSON API / HTTP endpoint you can use curl in a simple inventory script as follows:

#!/usr/bin/env sh

inventory_endpoint='https://example.com/inventory.json'
hostvars_endpoint='https://example.com/inventory/hosts/%s/hostvars.json'

http_user='user'
http_password='password'
http_endpoint="${inventory_endpoint}"

output_stdout=
validation_error=
exit_code=1

if ! command -v 'curl' &> /dev/null
then
  validation_error='curl is required to query the HTTP inventory endpoint!'
fi

if [ "$#" -ne 1 ] && [ "$#" -ne 2 ]
then
  validation_error="Usage: '$0 --list' or '$0 --host <hostname>'"
fi

if [ "$1" != '--list' ] && [ "$1" != '--host' ]
then
  validation_error="Invalid argument '$1'!"
fi

if [ "$1" != '--list' ] && [ "$#" -ne 1 ]
then
  validation_error='The --list flag does not take any arguments!'
fi

if [ "$1" != '--host' ] && [ "$#" -ne 2 ]
then
  validation_error='Missing argument for --host parameter!'
fi

if [ "$1" = '--host' ]
then
  http_endpoint=$(printf '..%s..' "${hostvars_endpoint}" "$2")
fi

if [ ! -z "${validation_error}" ]
then
  echo "${validation_error}" >&2
else 
  output_stdout=$(curl --silent --fail --header 'Accept: application/json' --user "${http_user}:${http_password}" "${http_endpoint}")
  exit_code="$?"
fi

if [ "${exit_code}" -ne 0 ]
then
  output_stdout='{}';
fi

echo "${output_stdout}";
exit "${exit_code}";

Example JSON API response (= STDOUT of <inventory_script> --list):

{
    "all": {
        "children": [
            "group1",
            "ungrouped"
        ],
    },
    "group1": {
        "hosts": {
            "host1.example.com": null,
            "host2.example.com": null
        },
        "vars": {
             "some_variable": "value_for_group1"
        }
    },
    "ungrouped": {
        "hosts": {
              " host3.example.com": null
        }
    },
    "_meta": {
        "hostvars": {
            "host3.example.com": {
                "some_variable" : "value_for_host3"
            }
        }
    },
}

Things to keep in mind

• The inventory script must accept --list and --host <hostname> as arguments.
• If the _meta key contains a value for “hostvars”, the inventory script will not be invoked with --host for each host.
• ansible calling --host for every host is rather inefficient - especially if it involves API calls to a remote subsystem.

Further:

If you intend to replace an existing static inventory file with an inventory script, it must return a JSON object which contains an all group that includes every host in the inventory as a member and every group in the inventory as a child. It should also include an ungrouped group which contains all hosts which are not members of any other group.

More detailed examples of the expected JSON format returned by an inventory can be found in the developer guide chapter Developing dynamic inventory.

[wenottingham]

The way inventory works is via Ansible inventory plugins - if one existed that fetched inventory via HTTP, it could be added to AWX and utilized. There isn't one at the moment.