include_filters without filters returns ALL hosts

Summary

include_filters is matching on ALL hosts when I expect it to match 0 hosts.

I think what is happening is that filters is the initial filter that is being applied and then include_filters are being applied to that set of matching hosts. If you don't specify ANY filters then it's equivalent to True as the initial filter. Then, when include_filters gets applied you get something like True || include_filter_1 || include_filter_2. Effectively, the include_filters become a NOOP if filters is not specified. I confirmed this suspicion with some pdb in the code.

https://github.com/ansible-collections/amazon.aws/blob/5876c14510420af190b309365a0843008696b602/plugins/inventory/aws_ec2.py#L712

include_filters = [self.get_option('filters')] + self.get_option('include_filters') <-- in my debugging this is [{}, {'key-name': 'jenkins'}]. Notice the first {}, when the filter list is passed to ansible_dict_to_boto3_filter_list(filter) the empty dict is equivalent to True.

https://github.com/ansible-collections/amazon.aws/pull/456 I have a potential fix here, but I feel like it may not take into account the exclude_filters feature.

Issue Type

Bug Report

Component Name

aws_ec2

Ansible Version

$ ansible --version
ansible [core 2.11.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /var/lib/awx/venv/awx/lib64/python3.8/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /var/lib/awx/venv/awx/bin/ansible
  python version = 3.8.6 (default, Jan 29 2021, 17:38:16) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]
  jinja version = 2.11.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0
ansible.netcommon             2.3.0
ansible.posix                 1.2.0
ansible.utils                 2.3.1
ansible.windows               1.7.2
arista.eos                    2.2.0
awx.awx                       19.2.2
azure.azcollection            1.8.0
check_point.mgmt              2.0.0
chocolatey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.2
cisco.intersight              1.0.16
cisco.ios                     2.3.1
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.8.0
community.digitalocean        1.8.0
community.docker              1.9.0
community.fortios             1.0.0
community.general             3.5.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.0
community.mysql               2.1.0
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.1.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.12.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.6.2
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.4.4
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.2.0
junipernetworks.junos         2.4.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.9.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.5.4
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.11.1
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.20.0
theforeman.foreman            2.1.2
vyos.vyos                     2.5.0
wti.remote                    1.0.1

# /var/lib/awx/venv/awx/lib64/python3.8/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0
ansible.netcommon             2.3.0
ansible.posix                 1.2.0
ansible.utils                 2.3.1
ansible.windows               1.7.2
arista.eos                    2.2.0
awx.awx                       19.2.2
azure.azcollection            1.8.0
check_point.mgmt              2.0.0
chocolatey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.2
cisco.intersight              1.0.16
cisco.ios                     2.3.1
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.8.0
community.digitalocean        1.8.0
community.docker              1.9.0
community.fortios             1.0.0
community.general             3.5.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.0
community.mysql               2.1.0
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.1.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.12.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.6.2
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.4.4
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.2.0
junipernetworks.junos         2.4.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.9.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.5.4
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.11.1
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.20.0
theforeman.foreman            2.1.2
vyos.vyos                     2.5.0
wti.remote                    1.0.1

# /var/lib/awx/.ansible/collections/ansible_collections
Collection Version
---------- -------
amazon.aws 1.5.0

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.18.18
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires: botocore, s3transfer, jmespath
Required-by:
---
Name: botocore
Version: 1.21.18
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed

OS / Environment

CentOS Linux release 8.4.2105

Steps to Reproduce

---
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - us-east-2
strict_permissions: False
include_filters:
- key-name: "does_not_exist"

ansible-inventory -i aws_ec2.yml --list

Do NOT use the filters: feature.
Make sure the include_filters does NOT match anything in your EC2 infrastructure.

Expected Results

I expected 0 hosts to be returned because I assumed that at least 1 filter in the include_filters list had to match.

Actual Results

All hosts in my EC2 infrastructure were returned.

Code of Conduct

[X] I agree to follow the Ansible Code of Conduct

ansible-collections / amazon.aws