Relative path in `private_data_dir` breaks inventory loading

[dmchmk]

Hi! In my project I have the following file structure:

root@b92f3c157e36:/code# tree
.
├── Dockerfile
├── Pipfile
├── Pipfile.lock
├── README.md
├── alembic.ini
├── app
│   ├── __init__.py
│   ├── helpers.py
│   ├── main.py
│   └── settings.py
└── playbooks
    ├── artifacts
    │   └── f10877d0-26c8-4ae0-b5c2-74c77fe63d1b
    │       ├── command
    │       ├── fact_cache
    │       ├── job_events
    │       │   ├── 1-2d02578c-3030-48ee-abb7-7a88d8f17be0.json
    │       │   ├── ...
    │       │   └── 9-c85150b7-5e5f-42f7-9b67-300d1f6887a6.json
    │       ├── rc
    │       ├── status
    │       ├── stderr
    │       └── stdout
    ├── check_server.yml
    ├── env
    │   └── ssh_key
    ├── id_rsa
    └── inventory
        └── hosts

And when in my code I pass private_data_dir as 'playbooks'

private_key = """-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
...
kI1C57WrbvfV8AAAAXbWVnYV92ZW5pa0BzdGl0Y2gubG9jYWwBAgME
-----END OPENSSH PRIVATE KEY-----
"""

inventory = """web1 ansible_host=1.0.0.1

[webservers]
web1 ansible_host=1.0.0.1
"""

r = ansible_runner.interface.run(
     ⦙   ⦙   host_pattern='all',
     ⦙   ⦙   private_data_dir='playbooks',
     ⦙   ⦙   playbook='check_server.yml',
     ⦙   ⦙   inventory=inventory,
     ⦙   ⦙   quiet=True,
     ⦙   ⦙   rotate_artifacts=1,
     ⦙   ⦙   ssh_key=private_key,
     ⦙   )

I get the following output:

  "Identity added: /code/playbooks/artifacts/9d78f285-b9d4-452a-a905-3faa85f6f04f/ssh_key_data (/code/playbooks/artifacts/9d78f285-b9d4-452a-a905-3faa85f6f04f/ssh_key_data)",
  "[WARNING]: Unable to parse /code/playbooks/playbooks/inventory/hosts as an",
  "inventory source",
  "[WARNING]: No inventory was parsed, only implicit localhost is available",
  "[WARNING]: provided hosts list is empty, only localhost is available. Note that",
  "the implicit localhost does not match 'all'",
  "",
  "\r\nPLAY [Update web servers] ******************************************************",
  "skipping: no hosts matched",
  "\r\nPLAY RECAP *********************************************************************"

Please, note, that ssh key was processed correctly via the correct path - /code/playbooks/artifacts/9d78f285-b9d4-452a-a905-3faa85f6f04f/ssh_key_data, but inventory path breaks and duplicates the private_data_dir value inside it - Unable to parse /code/playbooks/playbooks/inventory/hosts which leads to fail of the run.

When I pass absolute path (/code/playbooks) to the private_data_dir parameter, everything works correctly.

ansible-runner version: 2.3.2

[Shrews]

I'm not certain how things are working if you pass an absolute path because I see a couple of things wrong with your example:

• Your playbook to execute should reside in a project subdirectory of your private_data_dir (see https://ansible-runner.readthedocs.io/en/stable/intro/#runner-input-directory-hierarchy for a complete example).
• You are passing a string as the inventory parameter to run(), so this is going to overwrite whatever you have in env/hosts. The error message you see is from ansible-playbook (not ansible-runner) and it doesn't like what you have supplied it for hosts.

[Shrews]

To be more specific, this is the real error:

"skipping: no hosts matched"

The WARNINGS you see from ansible-playbook are generated as it attempts to go through various plugins to read the hosts file, that's normal. So it is somehow attempting to run some playbook where your target node is not matching your supplied hosts, but as you don't show me a project directory, I'm wondering if we're missing some key info here.

[dmchmk]

You are passing a string as the inventory parameter to run(), so this is going to overwrite whatever you have in env/hosts.

Yes, my first-place goal is to run Ansible from python-based webapp, using runtime-loaded variables like inventory and ssh-keys. So basically, the only file I've created by hand is /code/paybooks/check_server.yml

Your playbook to execute should reside in a project subdirectory of your private_data_dir (see https://ansible-runner.readthedocs.io/en/stable/intro/#runner-input-directory-hierarchy for a complete example).

Yes, really, I've missed this point, but looks like runner for some reason catches the right playbook file that I'm passing to it

But if I fix this issue, create project dir inside playbooks and move check_server.yml there, I'm getting the following output:

  "Identity added: /code/playbooks/artifacts/b4748e70-e20a-4a32-a401-e5addfdbcc57/ssh_key_data (/code/playbooks/artifacts/b4748e70-e20a-4a32-a401-e5addfdbcc57/ssh_key_data)",
  "[WARNING]: Unable to parse /code/playbooks/project/playbooks/inventory/hosts as",
  "an inventory source",
  "[WARNING]: No inventory was parsed, only implicit localhost is available",
  "[WARNING]: provided hosts list is empty, only localhost is available. Note that",
  "the implicit localhost does not match 'all'",
  "",
  "\r\nPLAY [Update web servers] ******************************************************",
  "skipping: no hosts matched",
  "\r\nPLAY RECAP *********************************************************************"

/code/playbooks/project/playbooks/ - why is it duplicating private_data_dir in path? New directory structure:

/code
├── Dockerfile
├── Pipfile
...
├── playbooks
│   ├── artifacts
│   │   └── b4748e70-e20a-4a32-a401-e5addfdbcc57
│   │       ├── command
│   │       ├── fact_cache
│   │       ├── job_events
│   │       │   ├── 1-fbc29fc2-7b3e-4ad4-8952-4debbcd064b3.json
│   │       │   ├── ...
│   │       │   └── 9-bf63c41e-19fe-4c10-953b-a057e97b059f.json
│   │       ├── rc
│   │       ├── status
│   │       ├── stderr
│   │       └── stdout
│   ├── env
│   │   └── ssh_key
│   ├── id_rsa
│   ├── inventory
│   │   └── hosts
│   └── project
│       └── check_server.yml

Playbook body:

---
- name: Update web servers
  hosts: all
  remote_user: root

  tasks:
  - name: Whoami
    command: whoami
    register: username

  - debug:
      msg: "{{ username.stdout }}"

Runner object:

r = ansible_runner.interface.run(
     ⦙   ⦙   private_data_dir="playbooks",
     ⦙   ⦙   playbook='check_server.yml',
     ⦙   ⦙   inventory=inventory,
     ⦙   ⦙   quiet=True,
     ⦙   ⦙   rotate_artifacts=1,
     ⦙   ⦙   ssh_key=private_key.private_key,
     ⦙   )

[Shrews]

It appears that there is indeed a bug when you set private_data_dir to a relative directory instead of a full path. I've been able to duplicate this locally. Thanks for identifying this. As we work to supply a fix, I suggest using a full path to work around this issue.

[dmchmk]

@Shrews much appreciate!