Pulumi CLI fails to use EC2 IAM role to configure AWS Provider

kerruba-milkman commented 1 year ago

What happened?

We are trying to run the Pulumi CLI from within an EC2 instance and we would like to leverage the EC2 IAM Role to perform the operations without passing any AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY credential. From within the EC2 instance I'm totally able to perform AWS operations using the SDK without the need to set a configuration file or credential file, and I can do the same with Terraform. Note that the IAM role has AdministratorAccess for testing purposes

When I try to run a Pulumi program using the Pulumi CLI from within the EC2 instance I get an error instead as Pulumi seems unable to correctly configure the Terraform Provider.

This is the command I'm calling

AWS_REGION=eu-central-1 pulumi preview -s <stack-name>

And this is the error I get:

Error: invocation of aws:index/getRegion:getRegion returned an error: 1 error occurred:
        * error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

    Please see https://registry.terraform.io/providers/hashicorp/aws
    for more information about providing credentials.

    Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, access disabled to EC2 IMDS via client option, or "AWS_EC2_METADATA_DISABLED" environment variable

This is an unexpected behavior since both the AWS CLI and the Terraform CLI works fine relying on the EC2 IAM Role to get temporary credentials.

Expected Behavior

Pulumi CLI is able to use the instance profile provided by EC2 to configure the Terraform Provider and perform the operations from within the EC2 machine without the need for a config/credential file or explicit AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY environment variables

Steps to reproduce

Create a Pulumi program
Start an EC2 instance with an associated profile having AdministratorAccess
Install all the necessary content
Run pulumi preview without passing any credential, but just relying on the AWS role inherited from the EC2 instance

Output of `pulumi about`

CLI
Version      3.67.1
Go Version   go1.20.4
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.21.1
awsx        1.0.0
command     0.7.1
docker      3.4.1
kafka       3.3.0
kubernetes  3.24.2
mysql       3.1.0
nodejs      unknown
postgresql  3.6.0
purrl       0.2.8
random      4.8.2
tls         4.6.1

Host
OS       amazon
Version  2
Arch     x86_64

This project is written in nodejs: executable='/home/ec2-user/.nvm/versions/node/v16.20.0/bin/node' version='v16.20.0'

Backend
Name           pulumi.com

Dependencies:
NAME                              VERSION
eslint-plugin-import              2.26.0
husky                             4.3.8
jest                              26.6.3
@pulumi/aws                       5.21.1
@pulumi/pulumi                    3.48.0
@pulumi/random                    4.8.2
@types/node                       18.13.0
eslint-config-airbnb-typescript   12.3.1
@types/lodash                     4.14.191
ts-jest                           26.5.6
@pulumi/awsx                      1.0.0
@pulumi/tls                       4.6.1
@types/prettier                   2.6.0
@typescript-eslint/parser         4.33.0
ts-node                           8.10.2
@types/amazon-cognito-auth-js     1.3.0
jose                              4.11.2
luxon                             3.2.1
@pulumi/kafka                     3.3.0
@pulumiverse/purrl                0.2.8
express                           4.18.1
lodash                            4.17.21
@types/express                    4.17.13
@typescript-eslint/eslint-plugin  4.33.0
eslint                            7.32.0
node-fetch                        3.3.0
typescript                        4.9.5
@pulumi/command                   0.7.1
@pulumi/kubernetes                3.24.2
@pulumi/mysql                     3.1.0
@pulumi/postgresql                3.6.0
@types/luxon                      3.2.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

justinvp commented 1 year ago

This issue looks specific to the AWS provider, so I'll transfer it to that repo and someone will take a look.

kerruba-milkman commented 1 year ago

Turned out this is not a bug but by default pulumi CLI skip the Metadata API and you need to turn the feature on for each AWS provider used in the program. This is documented here: https://www.pulumi.com/registry/packages/aws/installation-configuration/#authenticating-via-ec2-instance-metadata

I'm closing the ticket

pulumi / pulumi-aws