ansible_runner doesn't parse valid hosts.json file

[iancote]

I'm creating an inventory dict in python and passing it to ansible-runner but it's failing.
Relevant code:

def ansible_node(node: NodeRegister):
    # Build inv
    inv = {
        'all': {
            'hosts': {
                node.node: {
                    'ansible_host': node.inet,
                    'ansible_user': 'ansible'
                }
            }
        }
    }

    r = ansible_runner.run(inventory=inv, private_data_dir='ansible', playbook='test.yml')

ansible-runner does write out a hosts.json file before it errors.

(venv) iancote@iancote-XPS-13-7390:~/git/sladash/ansible$ cat inventory/hosts.json 
{"all": {"hosts": {"9222b13b3bfd": {"ansible_host": "192.168.42.129", "ansible_user": "ansible"}}}}

To further test, I tried running ansible-runner from cli using the created hosts.json, but get the same error:

(venv) iancote@iancote-XPS-13-7390:~/git/sladash$ ansible-runner run --inventory ansible/inventory/hosts.json -private_data_dir ansible --playbook test.yml
[WARNING]: Unable to parse
/home/iancote/git/sladash/ansible/ansible/inventory/hosts.json as an inventory
source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [Update node] *************************************************************
skipping: no hosts matched

PLAY RECAP *********************************************************************

However, if I just try calling ansible direct with the hosts.json from before, it works?

(venv) iancote@iancote-XPS-13-7390:~/git/sladash/ansible$ ansible -i inventory/hosts.json all -m ping 
9222b13b3bfd | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

Version info:

$ ansible --version
ansible [core 2.15.0]
  config file = None
  configured module search path = ['/home/iancote/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/iancote/git/sladash/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /home/iancote/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/iancote/git/sladash/venv/bin/ansible
  python version = 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] (/home/iancote/git/sladash/venv/bin/python)
  jinja version = 3.1.2
  libyaml = True
$ ansible-runner --version
2.3.3

Is there some mojo I'm missing here to make this work?

Thanks, -ian

[ishan-siddiqui]

The issue is that you are passing the inventory dictionary to ansible-runner as a string, but ansible-runner expects a file path. You can try to fix this by changing the following line of code:

r = ansible_runner.run(inventory=inv, private_data_dir='ansible', playbook='test.yml')

to

with open('hosts.json', 'w') as f:
    json.dump(inv, f, indent=4)

r = ansible_runner.run(inventory='hosts.json', private_data_dir='ansible', playbook='test.yml')

This will create a file called hosts.json in the current directory, and then pass the path to that file to ansible-runner. Once you have made this change, ansible-runner should be able to successfully parse the inventory and run the playbook.

Here is a more detailed explanation of the issue. When you pass a dictionary to ansible-runner as a string, ansible-runner tries to parse the string as an inventory source. However, the string you are passing is not a valid inventory source. An inventory source must be a file that contains a list of hosts, along with any associated variables. In your case, the string you are passing does not contain a list of hosts. It only contains a dictionary that defines a single host. As a result, ansible-runner is unable to parse the string as an inventory source.

When you create a file called hosts.json and write the inventory dictionary to that file, you are creating a valid inventory source. This is because the file contains a list of hosts, along with any associated variables. As a result, ansible-runner is able to successfully parse the file as an inventory source.

I hope this helps!

[iancote]

Thank you for taking a look Ishan. Per the docs for ansible_runner's inventory:

Overrides the inventory directory/file (supplied at private_data_dir/inventory) with a specific host or list of hosts. This can take the form of:
 - Path to the inventory file in the private_data_dir
 - Native python dict supporting the YAML/json inventory structure
 - A text INI formatted string
 - A list of inventory sources, or an empty list to disable passing inventory

when I run ansible_runner and pass it that simple inventory dict, it created the inventory directory and placed the hosts.json file there. It then failed to be able to use it.

Since I can use that same ansible_runner generated hosts.json file with ansible directly, that indicates the format of the file is fine and the relevant data is in place.

I've seen bugs filed where ansible_runner's generated inventory files were written with improper permissions and not readable for subsequent runs, but I verified the permissions on the hosts.json file and they're correct. So at this point I'm a little mystified as to what the breakdown is since ansible_runner's error message is a little cryptic.

[ishan-siddiqui]

I see. It sounds like you have already done a good job of troubleshooting the issue. The fact that you can use the same hosts.json file with Ansible directly indicates that the file is valid and that the permissions are correct.

I have looked at the Ansible-runner documentation and I can see that the inventory parameter can take a number of different forms, including a Python dictionary. However, I can't find any documentation that specifically says that the dictionary must be a valid Ansible inventory.

It is possible that the Ansible-runner developers have implemented some additional checks on the inventory dictionary, beyond simply checking that it is a valid JSON file. These checks may be failing for some reason.

[iancote]

Thanks Ishan - that sent me to look at how the plugins verify the information. From that I realized I was using 'private_data' wrong and needed to be using 'project_dir' instead. The code now works as expected.

I appreciate your inputs on this.