AWX 15.0.1: AWS EC2 Dynamic Source Inventory is broken

nicolas-g commented 4 years ago

ISSUE TYPE

Bug Report

SUMMARY

Using AWS secrets or IAM Roles in AWS and AWX version 14.1.0 (Ansible 2.9.11) we are able to start the sync process and use dynamic inventories.

It seems the exact same setup with AWX version 15.0.1, the dynamic source inventory is broken. The job sync fails and the UI has changed (see the screenshots).

ENVIRONMENT

AWX version: 15.0.1
AWX install method: Kubernetes
Ansible version: 2.9.14
Operating System: Centos
Web Browser: Chrome

STEPS TO REPRODUCE

Under Inventories -> Click on an existnig inventory -> Click on Sources tab -> Create a new source -> Choose Source Amazon EC2 -> Save

EXPECTED RESULTS

When you start a Sync Process job you should see hosts to be populated in your dynamic inventory.

ACTUAL RESULTS

Sync Process job fails

2020-11-10 22:09:19,903 INFO     awx.main.commands.inventory_import Updating inventory 4: dynamic inventory ng-xspdev TowerManaged tag
2020-11-10 22:09:19,913 INFO     awx.main.commands.inventory_import Reading Ansible inventory source: /tmp/awx_4_ucw29p7p/aws_ec2.yml
2020-11-10 22:09:19,914 INFO     awx.main.commands.inventory_import Using VIRTUAL_ENV: /var/lib/awx/venv/ansible
2020-11-10 22:09:19,914 INFO     awx.main.commands.inventory_import Using PATH: /var/lib/awx/venv/ansible/bin:/var/lib/awx/venv/awx/bin:/usr/pgsql-10/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2020-11-10 22:09:19,914 INFO     awx.main.commands.inventory_import Using PYTHONPATH: /var/lib/awx/venv/ansible/lib/python3.6/site-packages:
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/bin/awx-manage", line 8, in <module>
    sys.exit(manage())
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/__init__.py", line 154, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1142, in handle
    raise exc
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1032, in handle
    venv_path=venv_path, verbosity=self.verbosity).load()
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 208, in load
    return self.command_to_json(base_args + ['--list'])
  File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 191, in command_to_json
    self.method, proc.returncode, stdout, stderr))
RuntimeError: ansible-inventory failed (rc=1) with stdout:
stderr:
ansible-inventory 2.9.14
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible-inventory
  python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Using /etc/ansible/ansible.cfg as config file
[WARNING]:  * Failed to parse /tmp/awx_4_ucw29p7p/aws_ec2.yml with auto plugin:
Failed to describe instances: An error occurred (UnauthorizedOperation) when
calling the DescribeInstances operation: You are not authorized to perform this
operation.
  File "/usr/lib/python3.6/site-packages/ansible/inventory/manager.py", line 280, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/inventory/auto.py", line 58, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/var/lib/awx/vendor/awx_ansible_collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 642, in parse
    results = self._query(regions, filters, strict_permissions)
  File "/var/lib/awx/vendor/awx_ansible_collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 529, in _query
    return {'aws_ec2': self._get_instances_by_region(regions, filters, strict_permissions)}
  File "/var/lib/awx/vendor/awx_ansible_collections/ansible_collections/amazon/aws/plugins/inventory/aws_ec2.py", line 441, in _get_instances_by_region
    raise AnsibleError("Failed to describe instances: %s" % to_native(e))
[WARNING]: Unable to parse /tmp/awx_4_ucw29p7p/aws_ec2.yml as an inventory
source
ERROR! No inventory was parsed, please check your configuration and options.

ADDITIONAL INFORMATION

INSTANCE FILTERS has been renamed to HOST FILTER

14.1.0

15.0.1 REGIONS has been completely removed ENABLE VARIABLE and ENABLE VALUE has been added.

wenottingham commented 4 years ago

Did you install using the AWX operator?

nicolas-g commented 4 years ago

no, I installed running the install.yml Ansible playbook.

wenottingham commented 4 years ago

You don't have an AWS credential attached. Try creating one with your AWS credentials. (Inventory updates without credentials only work if you have assigned specific IAM roles to the instance.)

nicolas-g commented 4 years ago

@wenottingham as described in the issue summary

Using AWS secrets or IAM Roles

I can confirm this is still an issue either using AWS secrets or IAM Roles. It works fine in 14.1.0 but it is broken in 15.0.1 .

blomquisg commented 4 years ago

@ikhan2010 this looks like an issue with the AWS collection plugin. Can someone on your team chase this down?

jillr commented 4 years ago

I'm not able to reproduce this on a freshly deployed 15.0.1 (using docker-compose locally) and AWS secrets, my inventory is successfully populated. @nicolas-g would it be possible for you to share the job output (on debug please) from one of your tests using secrets or an IAM profile?

sveerabathini commented 1 year ago

@jillr, can you please provide me the setup details for IAM role, please

ansible / awx