aws sm and smp datasources fail on EC2 instance

[cofiem]

$ gomplate --version
gomplate version 4.1.0

$ aws --version
aws-cli/2.17.13 Python/3.11.9 Linux/5.15.0-1064-aws exe/x86_64.ubuntu.20

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

aws+sm

Command:

gomplate --verbose -d 'awsDs=aws+sm:' -i '{{ ds "awsDs" "myitemname" }}'

Output:

16:42:01 DBG starting gomplate
16:42:01 DBG config is:
---
datasources:
  awsDs:
    header: {}
    url: 'aws+sm:'
in: '{{ ds "a...'
 version=4.1.0 build=cc2584028866967a39b096265d5b9af4516c734f
16:42:01 DBG completed rendering templatesRendered=0 errors=1 duration=4.947283ms
16:42:01 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"awsDs\" \"myitemname\">: error calling ds: couldn't read datasource 'awsDs' (aws+sm:///myitemname): stat (url: \"aws+sm:///\", name: \"myitemname\"): stat myitemname: getSecretValue: operation error Secrets Manager: GetSecretValue, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region"

What I think is the equivalent aws cli command does work:

aws secretsmanager get-secret-value --secret-id myitemname

Output:

{ "ARN": "arn:aws:secretsmanager:...", "Name": "myitemname", "SecretString": "[expected content]"}

aws+smp

Command:

gomplate --verbose -d 'awsDs=aws+smp:' -i '{{ ds "awsDs" "myitemname" }}'

Output:

16:43:00 DBG starting gomplate
16:43:00 DBG config is:
---
datasources:
  awsDs:
    header: {}
    url: 'aws+smp:'
in: '{{ ds "a...'
 version=4.1.0 build=cc2584028866967a39b096265d5b9af4516c734f
16:43:00 DBG completed rendering templatesRendered=0 errors=1 duration=4.03562ms
16:43:00 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"awsDs\" \"myitemname\">: error calling ds: couldn't read datasource 'awsDs' (aws+smp:///myitemname): stat (url: \"aws+smp:///\", name: \"myitemname\"): stat myitemname: getParameter: operation error SSM: GetParameter, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region"

What I think is the equivalent aws cli command does work:

aws ssm get-parameter --name myitemname

Output:

{ "Parameter": { "ARN":"arn:aws:ssm:...", "Name": "myitemname", "Value": "[expected content]"} }

[cofiem]

Edit: It looks like it might be neccessary to explicitly get the region from the EC2 instance metadata? https://github.com/aws/aws-sdk-go-v2/issues/1967#issuecomment-1372867356

This might also be relevant?

https://github.com/go-acme/lego/issues/2067#issuecomment-1845666491

[hairyhenderson]

Thanks for posting those links - this sounds familiar and indeed the region is likely required now. As a workaround you can set the AWS_DEFAULT_REGION environment variable.

The fix for this (getting region from the EC2 IMDS) will need to go into go-fsimpl and this'll likely affect the other AWS filesystems (awssmfs, and awsimdsfs) too.

[cofiem]

Thanks for the workaround 🎉