aws_secret lookup does not accept ARN

[reskin89]

Summary

When using the aws_secret lookup plugin, it seems it only accepts a secret name, but not a full ARN, resulting in a failure to read cross account secrets.

Issue Type

Bug Report

Component Name

aws_secret

Ansible Version

$ ansible --version
ansible [core 2.14.1]
  config file = None
  configured module search path = ['/Users/reskin011/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/reskin011/.ansible/collections:/usr/share/ansible/collections
  executable location = /Library/Frameworks/Python.framework/Versions/3.11/bin/ansible
  python version = 3.11.0 (v3.11.0:deaf509e8f, Oct 24 2022, 14:43:23) [Clang 13.0.0 (clang-1300.0.29.30)] (/Library/Frameworks/Python.framework/Versions/3.11/bin/python3)
  jinja version = 3.1.2
  libyaml = False

Collection Versions

$ ansible-galaxy collection list
community.aws                 5.0.0  

AWS SDK versions

$ pip show boto boto3 botocore

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.26.47
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: aws-sam-cli, aws-sam-translator, serverlessrepo
---
Name: botocore
Version: 1.29.47
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

"{{ lookup( 'aws_secret', 'arn:aws:secretsmanager:us-east-1:{{ aws_account_id }}:secret:my-super-secret-password', region=secrets_region)  }}"

OS / Environment

Amazon Linux 2 and MacOS

Steps to Reproduce

1. Create a secret in an AWS account
2. provide a role and policy to allow access to that secret in account 2
3. use the playbook against a host with the proper role in account 2 with the below lookup call, replacing my-super-secret-password with the alias/name of your secret in the initial account

"{{ lookup( 'aws_secret', 'arn:aws:secretsmanager:us-east-1:{{ aws_account_id }}:secret:my-super-secret-password', region=secrets_region)  }}"

Expected Results

I expect the secret to be read and value rendered, instead a permission denied replies.

To ensure the instance can read the secret, running aws secretesmanager get-secret-value --secret-id FULLARN works without issue from the instance in question.

Actual Results

fatal: [REDACTED]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'aws_secret'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Failed to access secret arn:aws:secretsmanager:us-east-1:REDACTED:secret:my-super-secret-password (AccessDenied). Failed to access secret arn:aws:secretsmanager:us-east-1:REDACTED:secret:my-super-secret-password (AccessDenied)"
}

But logging into this instance and using the aws CLI:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-east-1:REDACTED:secret:my-super-secret-password  --region us-east-1
{
    "Name": "my-super-secret-password",
    "VersionId": "9FBB258E-7D9F-4C88-8FD9-8EDFC24B9E2B",
    "SecretString": "REDACTED",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": 1690903868.277,
    "ARN": "arn:aws:secretsmanager:us-east-1:REDACTED:secret:my-super-secret-password-dXJZy5"
}

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description: None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[andycui66]

Same on collection amazon.aws 6.4.0 aws_secret lookup does not support cross-account secrets.

[kyrylomiro]

any progress on this?

[surry355f1]

I need to reference secrets cross account as i dont want to have them stored in each account (hard to managed that way). Any progress ? or workaround?

[tremble]

@surry355f1,

I just tried a basic test of looking up a secret by ARN in our CI, and can't reproduce the issue ( See #2211 the test logs are at https://f06bb1c0876a2035eb30-0541d2f937b216ef65e5a86231415176.ssl.cf2.rackcdn.com/2211/fd522070620168012fe4afade452eaf434f4e3e4/check/integration-amazon.aws-1/2cd1ff0/job-output.txt )

I've also performed a quick cross-account test (not possible inside our CI) and I'm able to access the secrets using this lookup module.

This would seem to imply that the issue is with the permissions (especially since the error given is AccessDenied), rather than with this lookup module itself. Please see https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_examples_cross.html for an example of the necessary permissions, which includes configuring permissions for both SecretsManager and KMS.