danielmanser commented 3 years ago

Summary

When I try to run a playbook that has community.aws.route53 in it, I get an exception from botocore: botocore.exceptions.NoCredentialsError: Unable to locate credentials, even though I have a correct ~/.aws/credentials file. I think I've configured everything correctly according to the docs.

Using boto3 in the same Python environment (pyenv) works perfect, it is able to find my credentials.

Issue Type

Bug Report

Component Name

route53

Ansible Version

$ ansible --version
ansible [core 2.11.4]
  config file = /Users/dmanser/git/ansible/ansible.cfg
  configured module search path = ['/Users/dmanser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/dmanser/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/dmanser/.pyenv/versions/ansible/bin/ansible
  python version = 3.9.4 (default, Sep  8 2021, 15:35:31) [Clang 12.0.5 (clang-1205.0.22.11)]
  jinja version = 3.0.1
  libyaml = False

Collection Versions

$ ansible-galaxy collection list
tey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.3
cisco.intersight              1.0.17
cisco.ios                     2.4.0
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.1
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.9.2
community.digitalocean        1.9.0
community.docker              1.9.1
community.fortios             1.0.0
community.general             3.6.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.1
community.mysql               2.1.1
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.2.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.13.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.7.0
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.3.0
junipernetworks.junos         2.5.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.10.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.6.2
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.12.0
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.21.1
theforeman.foreman            2.2.0
vyos.vyos                     2.5.1
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.18.40
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages
Requires: jmespath, botocore, s3transfer
Required-by:
---
Name: botocore
Version: 1.21.40
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages
Requires: jmespath, urllib3, python-dateutil
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed
DEFAULT_BECOME(/Users/dmanser/git/ansible/ansible.cfg) = True
DEFAULT_BECOME_ASK_PASS(/Users/dmanser/git/ansible/ansible.cfg) = True
DEFAULT_FORKS(/Users/dmanser/git/ansible/ansible.cfg) = 10
DEFAULT_HOST_LIST(/Users/dmanser/git/ansible/ansible.cfg) = ['/Users/dmanser/git/ansible/inventory/test/hosts.yml']
DEFAULT_INTERNAL_POLL_INTERVAL(/Users/dmanser/git/ansible/ansible.cfg) = 0.001
DEFAULT_REMOTE_USER(/Users/dmanser/git/ansible/ansible.cfg) = dmanser
DEFAULT_ROLES_PATH(/Users/dmanser/git/ansible/ansible.cfg) = ['/Users/dmanser/git/ansible/roles']
DEFAULT_TRANSPORT(/Users/dmanser/git/ansible/ansible.cfg) = ssh
DEFAULT_VAULT_PASSWORD_FILE(/Users/dmanser/git/ansible/ansible.cfg) = /Users/dmanser/git/ansible/.vaultpassword
HOST_KEY_CHECKING(/Users/dmanser/git/ansible/ansible.cfg) = False
RETRY_FILES_SAVE_PATH(/Users/dmanser/git/ansible/ansible.cfg) = /tmp

OS / Environment

macOS Big Sur 11.5.2 on arm64 (Apple M1 silicon)

Darwin guybrush 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:27 PDT 2021; root:xnu-7195.141.2~5/RELEASE_ARM64_T8101 arm64

Steps to Reproduce

Setup

~/.aws/credentials

[default]
aws_access_key_id = AKXXXXXXXXXXXXXX
aws_secret_access_key = oRXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

~/.boto

[Credentials]
aws_access_key_id = AKXXXXXXXXXXXXXX
aws_secret_access_key = oRXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

roles/route53/tasks/main.yml

Note that I want to run the AWS command locally, hence the delegate_to. For simplicity, I've hard-coded the values in the main.yml file to rule out any other issues (i.e. incorrect vars).

---
- name: Ensure A record
  community.aws.route53:
    state: present
    zone: mydomain.net
    record: lisa.mydomain.net
    type: A
    ttl: 60
    value: 127.0.0.1
    wait: no
  delegate_to: localhost

site.yml

[...]
- name: apply route53 role
  hosts: all
  roles:
    - route53
[...]

inventory/test/hosts.yml

---
all:
  hosts:
    lisa.mydomain.net:
  children:
    kvmhosts:
      hosts:
        lisa.mydomain.net:
      vars:
        libvirt_users:
          - dmanser

Command used: ansible-playbook -i inventory/test/hosts.yml site.yml. This fails with the result listed below.

Also, I've ran boto3 within the same folder and environment, which works:

Also, setting the AWS Key ID and Secret Key directly in the role worked.

Expected Results

I expected community.aws.route53 will use my credentials stored in ~/.boto file, documented here. However, it seems not to find either ~/.aws/credentials, ~/.boto, nor if I set my AWS credentials in environment variables.

Actual Results

BECOME password:

PLAY [apply common role to all nodes] **********************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [lisa.mydomain.net]

[... other steps ...]

PLAY [apply route53 role] **********************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [lisa.mydomain.net]

TASK [route53 : Ensure A record] ***************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.NoCredentialsError: Unable to locate credentials
fatal: [lisa.mydomain.net -> localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/Users/dmanser/.ansible/tmp/ansible-tmp-1631385623.937531-62907-280358124821110/AnsiballZ_route53.py\", line 100, in <module>\n    _ansiballz_main()\n  File \"/Users/dmanser/.ansible/tmp/ansible-tmp-1631385623.937531-62907-280358124821110/AnsiballZ_route53.py\", line 92, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/dmanser/.ansible/tmp/ansible-tmp-1631385623.937531-62907-280358124821110/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.route53', _modlib_path=modlib_path),\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/lib/python3.9/runpy.py\", line 210, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/lib/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/lib/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 698, in <module>\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 587, in main\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 408, in get_zone_id_by_name\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py\", line 157, in retry_func\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py\", line 144, in retry_func\n  File \"/tmp/ansible_community.aws.route53_payload_44v8eefn/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 386, in _list_hosted_zones\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/paginate.py\", line 449, in build_full_result\n    for response in self:\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/paginate.py\", line 255, in __iter__\n    response = self._make_request(current_kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/paginate.py\", line 332, in _make_request\n    return self._method(**current_kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/client.py\", line 386, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/client.py\", line 691, in _make_api_call\n    http, parsed_response = self._make_request(\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/client.py\", line 711, in _make_request\n    return self._endpoint.make_request(operation_model, request_dict)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/endpoint.py\", line 102, in make_request\n    return self._send_request(request_dict, operation_model)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/endpoint.py\", line 132, in _send_request\n    request = self.create_request(request_dict, operation_model)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/endpoint.py\", line 115, in create_request\n    self._event_emitter.emit(event_name, request=request,\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/hooks.py\", line 357, in emit\n    return self._emitter.emit(aliased_event_name, **kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/hooks.py\", line 228, in emit\n    return self._emit(event_name, kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/hooks.py\", line 211, in _emit\n    response = handler(**kwargs)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/signers.py\", line 90, in handler\n    return self.sign(operation_name, request)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/signers.py\", line 162, in sign\n    auth.add_auth(request)\n  File \"/Users/dmanser/.pyenv/versions/3.9.4/envs/ansible/lib/python3.9/site-packages/botocore/auth.py\", line 373, in add_auth\n    raise NoCredentialsError()\nbotocore.exceptions.NoCredentialsError: Unable to locate credentials\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP *************************************************************************************************************
lisa.mydomain.net              : ok=7    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Code of Conduct

[X] I agree to follow the Ansible Code of Conduct

ansibullbot commented 3 years ago

Files identified in the description:

[plugins/modules/route53.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/route53.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 3 years ago

cc @jillr @jimbydamonk @markuman @s-hertel @tremble @wimnat click here for bot help

danielmanser commented 3 years ago

Found out why! This cost me hours, and right after I've posted this bug I realized I was running it with sudo locally, and root doesn't have the AWS credential file. I've added the become: false and it is working now:

---
- name: Ensure A record
  community.aws.route53:
    state: present
    zone: mydomain.net
    record: lisa.mydomain.net
    type: A
    ttl: 60
    value: 127.0.0.1
    wait: no
  delegate_to: localhost
  become: false

Silly me.

vadirajks commented 2 years ago

@danielmanser thanks , it helped me with your response!!.

SafeEval commented 2 years ago

This helped save me hours. Would be nice to include in the README to make the information more accessible.

ricardoandre97 commented 1 year ago

@danielmanser Just ran into the same problem. You're a rockstar!

ansible-collections / community.aws

NoCredentialsError: Unable to locate credentials #708