Variable stays in awx inventory even it is deleted in git repo.

[kzinas-adv]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.

Summary

Using sourced from project inventory update group_vars/all.yaml variables are not deleted After removing variable in git repo group_vars/all.yaml , it stays in awx default groups vars although it is gone. Change value of variable - OK Specific groups variables are not effected by this Specific host vars are not effected by this.

Inventory has these "Enabled Options": Overwrite local groups and hosts from remote inventory source Overwrite local variables from remote inventory source Update on project update

diff --git a/inventory/avv-prod/group_vars/all.yaml b/inventory/avv-prod/group_vars/all.yaml
index af2262cf..2a7c90de 100644
--- a/inventory/avv-prod/group_vars/all.yaml
+++ b/inventory/avv-prod/group_vars/all.yaml
@@ -52,4 +52,3 @@ git_dest: "{{ lookup('env', 'PWD') }}"
 grafana_url: "https://grafana.host.zz"
 grafana_user: "admin"
 grafana_password: "123456"
-test_var: test_zzz

After project refresh (+inventory update on dependency):

https://awx-test.host.zz/#/inventories/inventory/8/details Inventories -> avv-prod -> Variables

<skiped>
grafana_url: "https://grafana.host.zz"
grafana_user: "admin"
grafana_password: "123456"
test_var: test_zzz

AWX version

19.5.1

Installation method

kubernetes

Modifications

yes

Ansible version

2.12.1

Operating system

Debian 10.11

Web browser

No response

Steps to reproduce

Install awx 19.5.1 using operator latest on Kubernetes Create inventory which is sourced from project using options:

 Overwrite local groups and hosts from remote inventory source
 Overwrite local variables from remote inventory source
 Update on project update

Add variable to inventory git repo group_vars/all.yaml Commit git, push git, update project on awx. Check your variable in awx inventory group all. Delete it in git Commit git, push git, update project on awx. Check your variable in awx inventory group all.

Expected results

It should be gone.

Actual results

It still exists.

Additional information

Custom-ee

(.local) [d@z1 ansible-builder]$ cat requirements.txt 
urllib3
boto3
influxdb
dnspython
netaddr
psutil
bcrypt
cryptography>=3.0
Jinja2==2.11.3
jmespath==0.10.0
paramiko
pyasn1
PyNaCl
python-dateutil
six
supervisor
virtualenv
netaddr
asn1crypto
babel
cffi
idna
markupsafe
ply
pycparser
pytz
pyyaml
ara
PyMySQL

(.local) [domas@x1 ansible-builder]$ cat requirements.yml 
---
collections:
  - community.general
  - ansible.posix
  - ansible.utils
  - community.aws
  - community.crypto
  - community.grafana
  - community.mysql
  - community.zabbix
  - community.crypto

[kzinas-adv]

Same issue and with awx 17.1

[AlanCoding]

Also reported in https://github.com/ansible/awx/issues/1997

[yaserdashti]

i can see the same issue in AWX 20.1.0 . any plan for fixing it?

[sugitk]

A customer has also reported regarding this issue. I understand the comment in #1997 though, we should provide a workaround or an option to remove the variables.

https://github.com/ansible/awx/issues/1997#issuecomment-398462633

[kzinas-adv]

You can purge inventory/myinv/group_vars/all.yaml data through AWX API:
PUT https://{{ awx_server }}/api/v2/inventories/{{ awx_inventory_group_id }}/variable_data/
And later run update that inventory, this will recreate inventory/myinv/group_vars/all.yaml
All this can be included in one workflow template.

[doubletwist13]

At the very least a toggle would be nice, (perhaps per-inventory source) to allow for group_vars/all vars to be fully reloaded or something.

[pat-s]

I just fell over this in AWX 22.1.0 and can't believe this is not yet solved in a better way.

The API workaround does not really help as the automated inventory sync happens before the actual workflow. If you remove the vars within that, you don't have any vars.

The easiest workaround for local roles seems to be to update conditions from when: <var> is defined to when: <var> is defined and <var> != "" in combination with <var>: "" in group_vars.

[chipselden]

I've also been encountering this recently and it repeatedly trips Ansible users up in my organization. The only workaround we've found is to stop the usage of all.vars in our inventories and instead create a new custom group, we call it global, that every host gets added to. We lose some control over variable precedence since the all group gets treated specially in that regard.

It would be nice to have the ability to set a toggle on the whole inventory that all.vars should get wiped on a full sync.

The merged issue above, #13731, does not seem to address this since we are not using constructed inventories.

Edit: If there's another way we should be leveraging the existing functionality so that this isn't an issue, I'd be willing to explore it.

[janorn]

This is an issue for us too.

[xtxerr]

Big issue.

(using a dedicated group is not a workaround because of ansible's variable precedence, the group "all" has an important purpose)

[CWollinger]

Still exists in 23.1.0

[kzinas-adv]

The API workaround does not really help as the automated inventory sync happens before the actual workflow. If you remove the vars within that, you don't have any vars.

It works, we are using it for years, you just add additional task before your playbook run in workflow to resync inventory which you have purged.