RFE Enable aws_ec2 inventory plugin for ansible 2.9+

[kdelee]

ISSUE TYPE

• Feature Idea

Breaking out final acceptance criteria for aws_ec2 plugin because won't ship with first PR for https://github.com/ansible/awx/issues/171

COMPONENT NAME

• API

SUMMARY

We want parity when using Amazon EC2 inventory source between scripts and plugins for:

• hostvars
• default grouping + group names that are generated by it
• groupby options group names that are generated by it
• hostnames
• source var config override options

The current known blockers are in the category of source vars (default is listed with key):

• [] 'nested_groups': 'False',
• [] 'pattern_exclude': None, # duplicated with instance filters
• [] 'pattern_include': None, # duplicated with instance filters
• [] 'iam_role': None, https://github.com/ansible/ansible/issues/54059
• [] 'stack_filters': 'False', # interacts with instance filters
• [] 'vpc_destination_variable': 'ip_address'
• [ ] 'expand_csv_tags': 'False',

No parity but we think we need to punt because it never worked in the first place:

• 'destination_variable': 'public_dns_name',
• 'hostname_variable': None

No parity because we enforce our own setting:

• 'replace_dash_in_groups': 'True',

Current known DONE items (or are implemented by other features like group_by):

• 'boto_profile': None
• 'group_by_ami_id': 'True',
• 'group_by_availability_zone': 'True',
• 'group_by_aws_account': 'False',
• 'group_by_instance_id': 'True',
• 'group_by_instance_state': 'False',
• 'group_by_instance_type': 'True',
• 'group_by_key_pair': 'True',
• 'group_by_platform': 'True',
• 'group_by_region': 'True',
• 'group_by_security_group': 'True',
• 'group_by_tag_keys': 'True',
• 'group_by_tag_none': 'True',
• 'group_by_vpc_id': 'True',
• 'regions': 'all', # duplicated with source regions
• 'regions_exclude': 'us-gov-west-1, cn-north-1', # duplicated with source regions

No parity because plugin is only EC2 and we only ever supported use of EC2 script for EC2 even though it really did more than that:

• 'group_by_route53_names': 'True',
• 'group_by_elasticache_cluster': 'True',
• 'group_by_elasticache_engine': 'True',
• 'group_by_elasticache_parameter_group': 'True',
• 'group_by_elasticache_replication_group': 'True',
• 'group_by_rds_engine': 'True',
• 'group_by_rds_parameter_group': 'True',
• 'route53': 'False' # was not supported in the group_by options
• 'route53_excluded_zones': '',
• 'route53_hostnames': None,
• 'rds': 'False',
• 'all_elasticache_clusters': 'False',
• 'all_elasticache_nodes': 'False',
• 'all_elasticache_replication_groups': 'False',
• 'all_rds_instances': 'False',
• 'elasticache': 'True',
• 'eucalyptus': 'False',
• 'eucalyptus_host': None,

???? Don't know whats up with these

• 'cache_max_age': '300',
• 'cache_path': '~/.ansible/tmp',
• 'all_instances': 'False',

ADDITIONAL INFORMATION

Will update with related issues if they arise for individual vars\ @AlanCoding , you know who you are

[AlanCoding]

I believe this issue is now unblocked, although we do not have a specific person assigned to it yet. With the merge of https://github.com/ansible/ansible/pull/41637, the remaining items like nested_groups just require some switches on the AWX side.

[JonTheNiceGuy]

I've raised an issue (#ansible/68506) on the ansible/ansible side of things, because there was an issue with the ec2.py dynamic inventory script, and I thought it was on their side... but it turns out this is part of a larger problem that the ec2.py script uses the boto library, which is depreciated, and there's an aws_ec2 plugin instead that should be being used. I have a recent install of AWX and the ec2.py script is the one which is used when using the ec2 inventory type. Is there a loose idea of when aws_ec2 should be the default option, and is there a way to use it today?

[AlanCoding]

The inventory plugin isn't in ansible/ansible anymore either. The updated plan is https://github.com/ansible/awx/pull/6283, since the plugin moved to a collection, we will ship that collection and use the version from it.

We have logic written to use the aws_ec2 plugin, and as pointed out in your Ansible issue, the form of the inputs are expressed in a test:

https://github.com/ansible/awx/blob/devel/awx/main/tests/data/inventory/plugins/ec2/files/aws_ec2.yml

But the decision was made to not enable this because of the shortcomings mentioned above in this issue.

You can add a aws_ec2.yml file to source control, and use it as an SCM inventory for now.

[AlanCoding]

being done in https://github.com/ansible/awx/pull/6283

[kdelee]

We are now using the plugin and our regression tests are running and passing -- calling this verified. Any follow up issues for options or small bugs will be tracked in new issues.