ec2_metadata_facts returns 401 if IMDSv2 token times out

[tonyswu]

Summary

ec2_metadata_facts sometimes returns 401 unauthorized if the IMDSv2 token times out.

Looking at code here https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/ec2_metadata_facts.py#L613, 60 seconds may be too short if there are a lot of metadata to be loaded. Couple of potential solutions:

1. The session duration is made configurable.
2. Session duration is hardcoded to a bigger value.
3. ec2_metadata_facts to retrieve specific set of keys (rather than the entire metadata).

Issue Type

Bug Report

Component Name

ec2_metadata_facts

Ansible Version

$ /usr/local/bin/ansible --version
ansible [core 2.15.12]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, May 16 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Collection Versions

$ usr/local/bin/ansible-galaxy collection list

# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    6.5.0
ansible.netcommon             5.3.0
ansible.posix                 1.5.4
ansible.utils                 2.12.0
ansible.windows               1.14.0
arista.eos                    6.2.2
awx.awx                       22.7.0
azure.azcollection            1.19.0
check_point.mgmt              5.1.1
chocolatey.chocolatey         1.5.1
cisco.aci                     2.8.0
cisco.asa                     4.0.3
cisco.dnac                    6.9.0
cisco.intersight              1.0.27
cisco.ios                     4.6.1
cisco.iosxr                   5.0.3
cisco.ise                     2.6.2
cisco.meraki                  2.17.0
cisco.mso                     2.5.0
cisco.nso                     1.0.3
cisco.nxos                    4.4.0
cisco.ucs                     1.10.0
cloud.common                  2.1.4
cloudscale_ch.cloud           2.3.1
community.aws                 6.4.0
community.azure               2.0.0
community.ciscosmb            1.0.7
community.crypto              2.16.1
community.digitalocean        1.24.0
community.dns                 2.6.4
community.docker              3.4.11
community.fortios             1.0.0
community.general             7.5.2
community.google              1.0.0
community.grafana             1.6.1
community.hashi_vault         5.0.1
community.hrobot              1.8.2
community.libvirt             1.3.0
community.mongodb             1.6.3
community.mysql               3.8.0
community.network             5.0.2
community.okd                 2.3.0
community.postgresql          2.4.3
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.11.0
community.sap                 1.0.0
community.sap_libs            1.4.1
community.skydive             1.0.0
community.sops                1.6.7
community.vmware              3.11.1
community.windows             1.13.0
community.zabbix              2.2.0
containers.podman             1.11.0
cyberark.conjur               1.2.2
cyberark.pas                  1.0.23
dellemc.enterprise_sonic      2.2.0
dellemc.openmanage            7.6.1
dellemc.powerflex             1.9.0
dellemc.unity                 1.7.1
f5networks.f5_modules         1.27.1
fortinet.fortimanager         2.3.0
fortinet.fortios              2.3.4
frr.frr                       2.0.2
gluster.gluster               1.0.2
google.cloud                  1.3.0
grafana.grafana               2.2.3
hetzner.hcloud                1.16.0
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.12.0
ibm.storage_virtualize        2.1.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.5.0
inspur.ispim                  1.3.0
inspur.sm                     2.3.0
junipernetworks.junos         5.3.1
kubernetes.core               2.4.0
lowlydba.sqlserver            2.2.2
microsoft.ad                  1.4.1
netapp.aws                    21.7.1
netapp.azure                  21.10.1
netapp.cloudmanager           21.22.1
netapp.elementsw              21.7.0
netapp.ontap                  22.8.3
netapp.storagegrid            21.11.1
netapp.um_info                21.8.1
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.15.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.1.0
ngine_io.vultr                1.1.3
openstack.cloud               2.2.0
openvswitch.openvswitch       2.1.1
ovirt.ovirt                   3.2.0
purestorage.flasharray        1.24.0
purestorage.flashblade        1.14.0
purestorage.fusion            1.6.0
sensu.sensu_go                1.14.0
servicenow.servicenow         1.0.6
splunk.es                     2.1.2
t_systems_mms.icinga_director 1.33.1
telekom_mms.icinga_director   1.35.0
theforeman.foreman            3.15.0
vmware.vmware_rest            2.3.1
vultr.cloud                   1.11.0
vyos.vyos                     4.1.0
wti.remote                    1.0.5

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.34.117
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: s3transfer, botocore, jmespath
Required-by:
---
Name: botocore
Version: 1.34.144
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: python-dateutil, urllib3, jmespath
Required-by: s3transfer, boto3, awscli

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_LOG_PATH(/etc/ansible/ansible.cfg) = /var/log/ansible/localhost.log

OS / Environment

Rocky9

Steps to Reproduce

- hosts: localhost
  become: yes
  connection: local
  tasks:
    - amazon.aws.ec2_metadata_facts:

    - debug: msg="{{ ansible_ec2_instance_id }}"

Expected Results

Expected to see variable such as ansible_ec2_instance_id populated.

Actual Results

fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to retrieve metadata from AWS: HTTP Error 401: Unauthorized", "response": {"body": "", "connection": "close", "content-length": "0", "content-type": "text/plain", "date": "Wed, 24 Jul 2024 21:54:04 GMT", "msg": "HTTP Error 401: Unauthorized", "server": "EC2ws", "status": 401, "url": "http://169.254.169.254/latest/meta-data/system"}}

To be clear, this is not a problem with the instance itself. Manually retrieving metadata works.

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[abhishek-yadav32024]

can we get any alternative solution for this as we are also facing similar issue.

[tonyswu]

I am just doing it manually right now. I really only need instance ID from metadata, and this is what I am doing:

- name: Get instance ID
  block:
    - shell: |
        TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 120")
        curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/instance-id
      register: metadata_curl_return
    - set_fact:
        ansible_ec2_instance_id: '{{ metadata_curl_return.stdout_lines[0] }}'

[alinabuzachis]

Hello @tonyswu @abhishek-yadav32024 will you be willing to propose a patch and eventually open a PR for this issue? Thanks.

[tonyswu]

Yeah, I'd be willing to work on a PR for this. I'll see if I can put some time into this over the weekend.

[tonyswu]

@alinabuzachis I've submitted pull request https://github.com/ansible-collections/amazon.aws/pull/2209 for this.