search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
309 stars 341 forks source link

String variables under compose in the inventory file are not applied to hosts #570

Closed rwhagw closed 5 months ago

rwhagw commented 2 years ago

Summary

When adding variables under compose in an inventory file, any dictionaries, lists, and tuples are applied to the hosts, but string variables are ignored.

Issue Type

Bug Report

Component Name

amazon.aws.aws_ec2 - EC2 inventory source

Ansible Version

$ ansible --version
ansible [core 2.12.0]
  config file = /home/me/repos/ansible/ansible.cfg
  configured module search path = ['/home/me/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /home/me/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.7 (default, Oct 10 2021, 15:13:22) [GCC 11.1.0]
  jinja version = 3.0.3
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /usr/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.1  
ansible.netcommon             2.4.0  
ansible.posix                 1.3.0  
ansible.utils                 2.4.2  
ansible.windows               1.7.3  
arista.eos                    2.2.0  
awx.awx                       19.4.0 
azure.azcollection            1.10.0 
check_point.mgmt              2.1.1  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.1.0  
cisco.asa                     2.1.0  
cisco.intersight              1.0.17 
cisco.ios                     2.5.0  
cisco.iosxr                   2.5.0  
cisco.meraki                  2.5.0  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.7.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.1.0  
community.crypto              1.9.6  
community.digitalocean        1.11.0 
community.docker              1.10.0 
community.fortios             1.0.0  
community.general             3.8.1  
community.google              1.0.0  
community.grafana             1.2.3  
community.hashi_vault         1.4.1  
community.hrobot              1.2.0  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.2  
community.mongodb             1.3.1  
community.mysql               2.3.1  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.5.0  
community.proxysql            1.3.0  
community.rabbitmq            1.1.0  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.15.0 
community.windows             1.7.0  
community.zabbix              1.5.0  
containers.podman             1.8.1  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.6.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.12.0 
fortinet.fortimanager         2.1.3  
fortinet.fortios              2.1.2  
frr.frr                       1.0.3  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.3.0  
junipernetworks.junos         2.6.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.6.0 
netapp.azure                  21.9.0 
netapp.cloudmanager           21.11.0
netapp.elementsw              21.6.1 
netapp.ontap                  21.12.0
netapp.um_info                21.7.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.3.0  
ngine_io.cloudstack           2.2.2  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.1  
openvswitch.openvswitch       2.0.2  
ovirt.ovirt                   1.6.4  
purestorage.flasharray        1.11.0 
purestorage.flashblade        1.7.0  
sensu.sensu_go                1.12.0 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.23.0 
theforeman.foreman            2.2.0  
vyos.vyos                     2.6.0  
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.20.7
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.23.7
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: s3transfer, boto3, awscli

Configuration

$ ansible-config dump --only-changed
ANSIBLE_PIPELINING(/home/me/ansible/ansible.cfg) = True
CACHE_PLUGIN(/home/me/ansible/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/me/ansible/ansible.cfg) = /tmp/ansible/
CACHE_PLUGIN_TIMEOUT(/home/me/ansible/ansible.cfg) = 3600
DEFAULT_FORKS(/home/me/ansible/ansible.cfg) = 10
DEFAULT_GATHERING(/home/me/ansible/ansible.cfg) = smart
DEFAULT_GATHER_SUBSET(/home/me/ansible/ansible.cfg) = ['all']
DEFAULT_HOST_LIST(/home/me/ansible/ansible.cfg) = ['/home/me/ansible/inv/aws']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/me/ansible/ansible.cfg) = True
DEFAULT_ROLES_PATH(/home/me/ansible/ansible.cfg) = ['/home/me/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/home/me/ansible/ansible.cfg) = yaml
DEFAULT_VAULT_PASSWORD_FILE(/home/me/ansible/ansible.cfg) = /home/me/ansible/vault
DISPLAY_SKIPPED_HOSTS(/home/me/ansible/ansible.cfg) = False
HOST_KEY_CHECKING(/home/me/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/home/me/ansible/ansible.cfg) = False

OS / Environment

Host OS: Arch Linux Target OS: CentOS 7.6

Steps to Reproduce

# in an aws_ec2.yml inventory file
plugin: aws_ec2
boto_profile: east_coast
cache: false
regions:
  - us-east-1
strict_permissions: False
hostnames:
  - tag:Name
strict: False
keyed_groups:
  - separator: ''
    key: tags.OS
  - separator: ''
    key: tags.ansible_group
  - separator: ''
    key: tags.CFGMGMT
filters:
  instance-state-name:
    - running
    - pending
compose:
  ansible_host: private_ip_address
  location_vars:
    location: "east_coast"
  location: "east_coast"

After running ansible-inventory --host HOSTNAME with the above inventory file, "location_vars.location" is set but "location" is not.

Expected Results

I expected "location" to be set as the string value defined in the inventory file.

Actual Results

...
    "launch_time": "2021-11-19T02:02:00+00:00",
    "location_vars": {
        "location": "east_coast"
    },
    "metadata_options": {
...

location is not in the dictionary of variables.

Code of Conduct

rwhagw commented 2 years ago

Looks like this might need to be addressed in the ansible core repo:

https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/inventory/__init__.py#L351

alinabuzachis commented 2 years ago

@rwhagw Thank you for raising this. The values in compose are Jinja expressions and the double quotes that you included are YAML syntax will go away when the file is parsed. Including quotes in the dict values, so that they are valid Jinja expressions, should solve your issue.

If I use location: "'east_coast'", I can see it in the result.

{
    "_meta": {
        "hostvars": {
            "instance-01": {
                ...
                "location": "east_coast",
                "location_vars": {
                    "location": "east_coast"
                },
                "maintenance_options": {
                    "auto_recovery": "default"
                },
...
adrian-arapiles commented 1 year ago

Hello, I can confirm the trick of "'whatever'" works perfectly with strings. By other hand, in my case, I have a more difficult string to append that it has quotes. For example to assign value to ansible_ssh_common_args variable.

I have a command that seems like -o StrictHostKeyChecking=no -o ProxyCommand="sh -c \"custom command\"" I tried several ways, even with {%raw%} custom command {% endraw %} with zero result.

Could you give me a clue about that?

Thanks 😄

rwhagw commented 1 year ago

I missed the notification for this; thank you @alinabuzachis, that works!

@adrian-arapiles maybe like this:

"'-o StrictHostKeyChecking=no -o ProxyCommand=\"sh -c \"custom command\"\"'"

GomathiselviS commented 5 months ago

It appears that this issue has been resolved in the latest release. If you believe otherwise, please feel free to reopen the issue.