pulumi / pulumi-awsx

AWS infrastructure best practices in component form!
https://www.pulumi.com/docs/guides/crosswalk/aws/
Apache License 2.0
223 stars 104 forks source link

Not possible to deny default providers in 1.0.0-beta11 #930

Open Oro opened 2 years ago

Oro commented 2 years ago

What happened?

Denying all default providers via the following stack configuration does not allow me to instantiate e.g. a VPC (I also tried it with a cloudtrail, presumably others also don't work).

config:
  pulumi:disable-default-providers:
    - "*"

Steps to reproduce

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";

const awsxProvider = new awsx.Provider('awsxprovider');
const awsProvider = new aws.Provider('awsprovider');

const vpc = new awsx.ec2.Vpc(
  'test-vpc', {}, {
    providers: [ awsxProvider, awsProvider ],
  });
{
  "name": "awsx-default-provider",
  "main": "index.ts",
  "devDependencies": {
    "@types/node": "^14"
  },
  "dependencies": {
    "@pulumi/pulumi": "^3.0.0",
    "@pulumi/aws": "^5.0.0",
    "@pulumi/awsx": "^1.0.0-beta"
  }
}
config:
  pulumi:disable-default-providers:
    - "*"

Expected Behavior

Pulumi should create the vpc with the supplied provider config. NB: With awsx 0.40 using the aws provider with the following works:

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";

const awsProvider = new aws.Provider('awsprovider');

const vpc = new awsx.ec2.Vpc(
  'test-vpc', {}, {
    provider: awsProvider,
  });
{
  "name": "awsx-default-provider",
  "main": "index.ts",
  "devDependencies": {
    "@types/node": "^14"
  },
  "dependencies": {
    "@pulumi/pulumi": "^3.0.0",
    "@pulumi/aws": "^5.0.0",
    "@pulumi/awsx": "^0.40"
  }
}
$ pulumi preview
Previewing update (bug):
     Type                                    Name                       Plan       
 +   pulumi:pulumi:Stack                     awsx-default-provider-bug  create     
 +   ├─ awsx:x:ec2:Vpc                       test-vpc                   create     
 +   │  ├─ awsx:x:ec2:NatGateway             test-vpc-0                 create     
 +   │  │  ├─ aws:ec2:Eip                    test-vpc-0                 create     
 +   │  │  └─ aws:ec2:NatGateway             test-vpc-0                 create     
 +   │  ├─ awsx:x:ec2:Subnet                 test-vpc-private-1         create     
 +   │  │  ├─ aws:ec2:Subnet                 test-vpc-private-1         create     
 +   │  │  ├─ aws:ec2:RouteTable             test-vpc-private-1         create     
 +   │  │  ├─ aws:ec2:RouteTableAssociation  test-vpc-private-1         create     
 +   │  │  └─ aws:ec2:Route                  test-vpc-private-1-nat-1   create     
 +   │  ├─ awsx:x:ec2:NatGateway             test-vpc-1                 create     
 +   │  │  ├─ aws:ec2:Eip                    test-vpc-1                 create     
 +   │  │  └─ aws:ec2:NatGateway             test-vpc-1                 create     
 +   │  ├─ awsx:x:ec2:Subnet                 test-vpc-public-0          create     
 +   │  │  ├─ aws:ec2:RouteTable             test-vpc-public-0          create     
 +   │  │  ├─ aws:ec2:Subnet                 test-vpc-public-0          create     
 +   │  │  ├─ aws:ec2:RouteTableAssociation  test-vpc-public-0          create     
 +   │  │  └─ aws:ec2:Route                  test-vpc-public-0-ig       create     
 +   │  ├─ awsx:x:ec2:InternetGateway        test-vpc                   create     
 +   │  │  └─ aws:ec2:InternetGateway        test-vpc                   create     
 +   │  ├─ awsx:x:ec2:Subnet                 test-vpc-private-0         create     
 +   │  │  ├─ aws:ec2:Subnet                 test-vpc-private-0         create     
 +   │  │  ├─ aws:ec2:RouteTable             test-vpc-private-0         create     
 +   │  │  ├─ aws:ec2:RouteTableAssociation  test-vpc-private-0         create     
 +   │  │  └─ aws:ec2:Route                  test-vpc-private-0-nat-0   create     
 +   │  ├─ awsx:x:ec2:Subnet                 test-vpc-public-1          create     
 +   │  │  ├─ aws:ec2:RouteTable             test-vpc-public-1          create     
 +   │  │  ├─ aws:ec2:Subnet                 test-vpc-public-1          create     
 +   │  │  ├─ aws:ec2:Route                  test-vpc-public-1-ig       create     
 +   │  │  └─ aws:ec2:RouteTableAssociation  test-vpc-public-1          create     
 +   │  └─ aws:ec2:Vpc                       test-vpc                   create     
 +   └─ pulumi:providers:aws                 awsprovider                create     

Resources:
    + 32 to create

Actual Behavior

$ pulumi preview
Previewing update (bug):
     Type                      Name                       Plan       Info
 +   pulumi:pulumi:Stack       awsx-default-provider-bug  create     1 error
 +   ├─ pulumi:providers:awsx  awsxprovider               create     
 +   └─ pulumi:providers:aws   awsprovider                create     

Diagnostics:
  pulumi:pulumi:Stack (awsx-default-provider-bug):
    error: Error: failed to register new resource test-vpc [awsx:ec2:Vpc]: 2 UNKNOWN: unknown provider 'urn:pulumi:denied::denied::denied$pulumi:providers:denied::awsx::denydefaultprovider'
        at Object.registerResource (/home/oro/bug/awsx-default-provider/node_modules/@pulumi/runtime/resource.ts:292:27)
        at new Resource (/home/oro/bug/awsx-default-provider/node_modules/@pulumi/resource.ts:402:13)
        at new ComponentResource (/home/oro/bug/awsx-default-provider/node_modules/@pulumi/resource.ts:890:9)
        at new Vpc (/home/oro/bug/awsx-default-provider/node_modules/@pulumi/ec2/vpc.ts:124:9)
        at Object.<anonymous> (/home/oro/bug/awsx-default-provider/index.ts:6:13)
        at Module._compile (node:internal/modules/cjs/loader:1119:14)
        at Module.m._compile (/home/oro/bug/awsx-default-provider/node_modules/ts-node/src/index.ts:439:23)
        at Module._extensions..js (node:internal/modules/cjs/loader:1173:10)
        at Object.require.extensions.<computed> [as .ts] (/home/oro/bug/awsx-default-provider/node_modules/ts-node/src/index.ts:442:12)
        at Module.load (node:internal/modules/cjs/loader:997:32)

Output of pulumi about

$ pulumi about
CLI          
Version      3.40.1
Go Version   go1.19
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.17.0
awsx    1.0.0-beta.11
docker  3.4.1
nodejs  unknown

Host     
OS       nixos
Version  22.11 (Raccoon)
Arch     x86_64

This project is written in nodejs: executable='/nix/store/zarkkci85li75a3rk1ssalcr3zvpn2j1-nodejs-18.9.1/bin/node' version='v18.9.1'

Backend        
Name           OroFramework
URL            file://./
User           oro
Organizations  

Dependencies:
NAME            VERSION
@pulumi/aws     5.17.0
@pulumi/awsx    1.0.0-beta.11
@pulumi/pulumi  3.43.1
@types/node     14.18.32

Pulumi locates its logs in /run/user/1000 by default
warning: Failed to get information about the current stack: No current snapshot
warning: A new version of Pulumi is available. To upgrade from version '3.40.1' to '3.43.1', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

guineveresaenger commented 2 years ago

Hi @Oro - thank you for opening this issue. We'll look into it as soon as we can.

danielrbradley commented 2 years ago

I think this might be another instance of the issue where explicit providers are not inherited correct - in the component framework - https://github.com/pulumi/pulumi/issues/10640

This will be able to be resolved once that issue is fixed and we can make a new release.

chris-stetter commented 1 year ago

Upstream issue seems resolved. Will this be in the next beta release?

danielrbradley commented 1 year ago

Yes, this should be fixed in the next release once we're upgraded dependencies. This should be happening quite soon.

chris-stetter commented 1 year ago

After upgrading to 1.0.0 I am still running into that issue. Can someone confirm?

danielrbradley commented 1 year ago

Are you testing this on a new or existing stack? An existing stacks may contain references to the old version of the awsx plugin from existing resources and might required a re-deploy to pull in the new provider version. If this is still persisting, we can raise this again in https://github.com/pulumi/pulumi

chris-stetter commented 1 year ago

The dependency versions are:

pulumi==3.47.2
pulumi-aws==5.17.0
pulumi-aws-native==0.40.2
pulumi-awsx==1.0.0
pulumi-docker==3.6.1
pulumi-random==4.8.2

I have set pulumi:disable-default-providers: ["*"] and deploy the following new stack to LocalStack:

import pulumi_awsx as awsx
import pulumi
import pulumi_aws as aws

provider = aws.Provider("provider")
awsx.ec2.Vpc("vpc", opts=pulumi.ResourceOptions(provider=provider))

Still results in Exception: Default provider for 'awsx' disabled. 'awsx:ec2:Vpc' must use an explicit provider.

justinvp commented 1 year ago

Here's a quick status update and workaround:

As of the 1.0.0 beta and later, AWSx is implemented as a component package so it can be used by all Pulumi languages, which means it has an associated awsx provider plugin.

As such, when using the feature to disable all default providers, you'll get an error when not specifying an explicit awsx provider for any AWSx components created in a stack: Default provider for 'awsx' disabled. 'awsx:ec2:Vpc' must use an explicit provider.

To address this, you'd have to specify an explicit awsx provider for the AWSx component, as well as an explicit aws provider for the component's children, something like:

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

awsxProvider = awsx.Provider("awsxProvider")
awsProvider = aws.Provider("awsProvider")

awsx.ec2.Vpc("vpc", opts=pulumi.ResourceOptions(provider=awsxProvider, providers=[awsProvider]))

Unfortunately, this doesn't currently work due to a limitation that prevents specifying an explicit provider for packaged components: https://github.com/pulumi/pulumi/issues/11520.

Workaround

In the meantime, the best workaround when using pulumi:disable-default-providers is to specify the list of providers rather than using the catch-all "*", e.g.:

config:
  pulumi:disable-default-providers:
    - aws
    - kubernetes

Since - awsx isn't listed, it won't error with Default provider for 'awsx' disabled..

And then specify the explicit provider to use for the component's children:

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

awsProvider = aws.Provider("awsProvider")
awsx.ec2.Vpc("vpc", opts=pulumi.ResourceOptions(provider=awsProvider)) # or: opts=pulumi.ResourceOptions(providers=[awsProvider])
chris-stetter commented 1 year ago

Will this be solved with https://github.com/pulumi/pulumi/pull/13282?

t0yv0 commented 1 month ago

Checking up on this issue, I understand that https://github.com/pulumi/pulumi/issues/11520 solves it for Python, TypeScript and Go, and Justin's example above should now be working. Leaving the issue in the backlog until all the languages are supported. Please let us know if this is not working for you as expected in the meanwhile.

Repeating Justin's snippet:

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

awsxProvider = awsx.Provider("awsxProvider")
awsProvider = aws.Provider("awsProvider")

awsx.ec2.Vpc("vpc", opts=pulumi.ResourceOptions(provider=awsxProvider, providers=[awsProvider]))