RFE: Make the value of a variable in a survey available in YAML in the inventory source

[akira6592]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.

Feature type

Enhancement to Existing Feature

Feature Summary

I would like to be able to reflect the values of additional variables (including surveys) in job templates and workflow job templates in the inventory source yaml.

The netbox.nebox.nb_inventory inventory plugin will be available in the next release with several filter options for extra vars. The purpose is to improve performance.

More info: https://github.com/netbox-community/ansible_modules/pull/834

In ansible-playbook command, the value of a variable given by the -e option can be used in the netbox.nebox.nb_inventory inventory plugin.

However, in AWX, the value of the variable does not seem to be passed on because the -e option is missing from the ansible-inventory command when retrieving the inventory. https://github.com/ansible/awx/blob/21.7.0/awx/main/tasks/jobs.py#L1506-L1539

Select the relevant components

• [X] UI
• [X] API
• [ ] Docs
• [ ] Collection
• [ ] CLI
• [ ] Other

Steps to reproduce

1. Prepare YAML for the inventory source as follows

   plugin: netbox.netbox.nb_inventory
   api_endpoint: https://demo.netbox.dev
   query_filters:
     - site: "{{ site }}"

2. In a job template or workflow job template, enter site variable value via Survey
3. Launch job

Current results

The following error message appears on Inventory Sync.

[WARNING]:  * Failed to parse /runner/project/nb_inventory.yml with auto
plugin: 'site' is undefined

Sugested feature result

1. The site variable in the inventory source's YAML reflects the value specified in the survey.
2. Only the hosts corresponding to the filter will be retrieved.
3. Job is executed for target host retrieved.

Additional information

n/a

[akira6592]

Any updates for this?

[akira6592]

Could be related #14868

[jseifeddine]

just chiming in as the OP of https://github.com/ansible/awx/issues/14868

I think this feature enhancement would be for ansible its self, not specifically AWX?

[akira6592]

@j-oss2023

I think AWX specific.

In ansible-playbook command, the value of a variable given by the -e option can be used in the netbox.nebox.nb_inventory inventory plugin.

[jseifeddine]

@akira6592 ineresting...

I made a custom dynamic plugin

https://github.com/j-oss2023/mysql-dynamic-inventory

And I'm unable to pass extra var to it ansible or awx ,

this gives me a clue that netbox inventory plugin is doing some extra stuff

Thanks !!!

I will investigate and fix my plugin so that it accepts extra var (if I can)

[jseifeddine]

Fixed my dynamic inventory plugin

https://github.com/j-oss2023/mysql-dynamic-inventory/commit/6a9a6aeebc97d63712a3643e1ac509b8779b2868?diff=unified&w=0

But - yes I am still experiencing the same exact issue when running it in AWX...

Works perfectly on cli - as per this example: https://github.com/ansible/awx/issues/14868#issuecomment-1945208879

But not in AWX :(

[akira6592]

Maybe it is the same as in my case.

[jseifeddine]

@akira6592

Yes, you're absolutely right - and thank you for your comment, you gave me a clue to fix my dynamic plugin - which now correctly uses -e, --extra-vars

    def parse(self, inventory, loader, path, cache=True):
        '''Parses the inventory file'''
        super(InventoryModule, self).parse(inventory, loader, path, cache)
        self._read_config_data(path)

        self.templar.available_variables = self._vars

        for option in self._options:
            try:
                if self.get_option(option):
                    templated_option = self.templar.template(self.get_option(option))
                    self.set_option(option, templated_option)
            except Exception as e:
                raise AnsibleError(f"Error processing templating for {option}: {str(e)}")

        self._fetch_hosts()

As you stated, this is an AWX problem,

ansible works fine from CLI

[jseifeddine]

@akira6592

Perhaps you are willing to implement this change - help me to figure out where it is, reverse engineer the awx repo

This is what podman reveals as the command for the inventory sync job:

ansible-inventory --list --export -i /runner/project/dynamic/all-mysql.yml --output /runner/artifacts/1842/output.json --playbook-dir /runner/project/dynamic -vvv

This is what podman reveals as the command for the playbook job:

ssh-agent sh -c trap 'rm -f /runner/artifacts/1845/ssh_key_data' EXIT && ssh-add /runner/artifacts/1845/ssh_key_data && rm -f /runner/artifacts/1845/ssh_key_data && ansible-playbook -u root -l myhost -i /runner/inventory -e @/runner/env/extravars myplaybook.yml

As you can see, the invocation of the inventory sync job omits the -e @/runner/env/extravars part - which is why we are having the issue we are having...

I'll start to debug / reverse engineer - see if i can make it sing...

If you have time to work on this let me know, Keen to implement and push a conformed fix 👍

[jseifeddine]

OK, i think its in awx/awx/main/tasks/jobs.py

BaseTask._write_extra_vars_file is involved...

and is called in classes RunJob, RunProjectUpdate and RunAdHocCommand within a function build_extra_vars_file

Still digging, but getting warmer

I'm by not means claiming to understand the wider implications of this, but gonna give it a good crack - see what i can do (with the help of ChatGPT)

Update on progress...

[jseifeddine]

Yeah - not going to be able to fix this... 📦

there's a lot to consider and change

The models: inventory, mixins ? the tasks: inventory

I'm probably only scratching the surface