Manually created provider doesn't use settings from pulumi config

dhilgarth commented 9 months ago

What happened?

I've configured my stack with the AWS credentials like so:

  aws:accessKey: AKREDACTEDMA
  aws:region: eu-central-1
  aws:secretKey:
    secure: v1:REDACTED:REDACTED

The implicit provider uses these credentials. However, if I manually create a new provider, it doesn't use them.
In an environment where the AWS CLI is configured with different credentials, it will silently use those and deploy to the wrong account. In an environment where the AWS CLI is not configured, it errors out:

 +   pulumi:pulumi:Stack      provider-test-dev  create
     └─ pulumi:providers:aws  usEast1                       1 error

Diagnostics:
  pulumi:providers:aws (usEast1):
    error: rpc error: code = Unknown desc = unable to validate AWS credentials.
    Details: No valid credential sources found. Please see https://www.pulumi.com/registry/packages/aws/installation-configuration/
    for more information about providing credentials.

Example

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

new aws.s3.Bucket("default-provider");

const usEast1Provider = new aws.Provider("usEast1", { region: "us-east-1" });
new aws.s3.Bucket("useast1-provider", {}, {provider: usEast1Provider});

Output of `pulumi about`

CLI Version 3.101.1 Go Version go1.21.5 Go Compiler gc

Plugins NAME VERSION aws 6.18.0 nodejs unknown

Host OS debian Version 12.4 Arch x86_64

This project is written in nodejs: executable='/usr/local/bin/node' version='v21.5.0'

Backend Name b3d6f4dc687c URL file://~ User root Organizations Token type personal

Dependencies: NAME VERSION @pulumi/aws 6.18.0 @pulumi/pulumi 3.101.1 @types/node 18.19.6

Additional context

My example is very simple and one could argue, why I'm doing this. There are a few resources that must be created in us-east-1 no matter what, e.g. certificates for cloudfront. So this is a common scenario: Same account, just different region.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 8 months ago

Hi @dhilgarth. This is the expected behavior of the explicit (in-code) providers. Part of the benefit they provide is to isolate your provider from ambient config. If you want to equip them with ambient credentials, you can pass values from your pulumi config to them on creation.

There is an issue to allow changing this behavior in pulumi/pulumi: https://github.com/pulumi/pulumi/issues/12176.

dhilgarth commented 8 months ago

That's the thing: it does use ambient config: environment variables, shared credentials file, EC2 Instance Metadata. The only thing it doesn't use is the pulumi config, even though the documentation makes it look like the pulumi config and environment variables are equivalent.

I've upvoted the other issue.

pulumi / pulumi-aws