aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.65k stars 3.91k forks source link

(aws-cdk): (Assets built in EKS/HELM/LAMBDA product in wrapped stack) #27311

Closed zosiek666 closed 1 year ago

zosiek666 commented 1 year ago

Describe the bug

On AWS, we are in the process of creating our custom product portfolio through Service Catalog. We wish to include the EKS product in our portfolio. Unfortunately, we encounter an error during the SYNTH and DEPLOY phases. Interestingly, when we attempt to deploy only the EKS product as the primary item, it functions as expected. However, when we try to include it as a product within Service Catalog, we encounter the aforementioned error. Regrettably, we have been unable to find a workaround for this issue. This problem also extends to HELMCharts and Lambdas (code defined in separate file). With Lambdas we have managed to work around the issue by using inline definitions, but still this is also a problem.

Expected Behavior

Synth OK and also successful deployment via CLI

Current Behavior

jsii.errors.JavaScriptError: 
  Error: An Asset Bucket must be provided to use Assets
      at ProductStackSynthesizer.addFileAsset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-servicecatalog/lib/private/product-stack-synthesizer.js:1:884)
      at NestedStackSynthesizer.addFileAsset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/core/lib/stack-synthesizers/nested.js:1:1025)
      at new Asset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1207)
      at AssetCode.bind (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4628)
      at new LayerVersion (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-lambda/lib/layers.js:1:2066)
      at new NodeProxyAgentLayer (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/lambda-layer-node-proxy-agent/lib/node-proxy-agent-layer.js:1:374)
      at new ClusterResourceProvider (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.js:1:698)
      at ClusterResourceProvider.getOrCreate (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.js:1:583)
      at new ClusterResource (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource.js:1:572)
      at new Cluster (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-MN2lmT/node_modules/aws-cdk-lib/aws-eks/lib/cluster.js:1:11473)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/xxx/service-catalog/app_eks.py", line 40, in <module>
    WrapperStack(app, "WrapperStackForEksProduct")
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__
    inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
  File "/Users/xxx/service-catalog/app_eks.py", line 36, in __init__
    EKSProduct(self, "EksProductInsideWrapper")
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__
    inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
  File "/Users/xxx/service-catalog/app_eks.py", line 15, in __init__
    cluster = eks.Cluster(self, 'EKSCluster',
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__
    inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
  File "/Users/xxx/lib/python3.9/site-packages/aws_cdk/aws_eks/__init__.py", line 13846, in __init__
    jsii.create(self.__class__, self, [scope, id, props])
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/__init__.py", line 334, in create
    response = self.provider.create(
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/providers/process.py", line 365, in create
    return self._process.send(request, CreateResponse)
  File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/providers/process.py", line 342, in send
    raise RuntimeError(resp.error) from JavaScriptError(resp.stack)
RuntimeError: An Asset Bucket must be provided to use Assets

Reproduction Steps

Create file with below python code and try do cdk-synth

import aws_cdk as cdk
from aws_cdk import (
    aws_eks as eks,
    aws_servicecatalog as sc,
    App,
)

from constructs import Construct

class EKSProduct(sc.ProductStack):
    def __init__(self, scope: Construct, id: str) -> None:
        super().__init__(scope, id)

        cluster = eks.Cluster(self, 'EKSCluster',
                              default_capacity=0,
                              version=eks.KubernetesVersion.V1_25,
                              cluster_name="MySimpleEKSCluster",
                              )

        # product = sc.CloudFormationProduct(self, "Product",
        #                                    product_name="My Product",
        #                                    owner="Product Owner",
        #                                    product_versions=[sc.CloudFormationProductVersion(
        #                                        product_version_name="v1",
        #                                        cloud_formation_template=sc.CloudFormationTemplate.from_product_stack(
        #                                            self),
        #                                    )]
        #                                    )

class WrapperStack(cdk.Stack):
    def __init__(self, scope: Construct, id: str) -> None:
        super().__init__(scope, id)

        EKSProduct(self, "EksProductInsideWrapper")

app = App()
WrapperStack(app, "WrapperStackForEksProduct")
app.synth()

Possible Solution

Allow to create assets for nested stack and wrapped stack, and pass ENV variables to them or allow to pass bucket where it cane be stored.

Additional Information/Context

Try on different python versions , CDK version, Node version and lib version. Also on different systems. Always the same result.

CDK CLI Version

2.98.0 (build b04f852)

Framework Version

aws-cdk-lib==2.98.0

Node.js Version

v18.16.0

OS

OSX 12.6.5, Windows 10, Windows 11

Language

Python

Language Version

Python 3.7.16, Python 3.8.0, Python 3.9.16

Other information

No response

peterwoodworth commented 1 year ago

Did you see the documentation on this module here?

Also,

cloud_formation_template=sc.CloudFormationTemplate.from_product_stack(self)

I'm not sure passing in the stack that contains the CloudFormation Product is supposed to be the pattern

zosiek666 commented 1 year ago

@peterwoodworth Hi, yes I saw that - but this is for lambda and cloudformation stack (where I have code for my lambda and can provide some assetes).

That is not the case here. I have update the example code (CF is not required to replicate issue - only EKS or HELMChart). Like you can see for EKS or HELMChart under the hood CDK is trying to deploy some assets. In my opinion problem is here (EKS documentation)

KubectlHandler - Lambda function for invoking kubectl commands on the cluster - created by CDK.
ClusterHandler - Lambda function for interacting with EKS API to manage the cluster lifecycle - created by CDK.

For HELMChart I think problem is when you want to create EKS cluster from_cluster_attributes.

peterwoodworth commented 1 year ago

Our EKS constructs require a place to put assets to be able to deploy them, so you would have to provide an asset bucket for the stack to upload assets to. Maybe I don't understand what you're trying to say...

zosiek666 commented 1 year ago

Ok so I will give you two examples. One is working, second no:

Example code: ```python import aws_cdk as cdk from aws_cdk import ( aws_eks as eks, App, ) from constructs import Construct class NormalStack(cdk.Stack): def __init__(self, scope: Construct, id: str) -> None: super().__init__(scope, id) cluster = eks.Cluster(self, 'EKSCluster', default_capacity=0, version=eks.KubernetesVersion.V1_25, cluster_name="MySimpleEKSCluster", ) app = App() NormalStack(app, "NormalStackForEksProduct") app.synth() ```
Response of cdk synth: ```yaml Resources: EKSClusterDefaultVpc85D27B27: Type: AWS::EC2::VPC Properties: CidrBlock: 10.0.0.0/16 EnableDnsHostnames: true EnableDnsSupport: true InstanceTenancy: default Tags: - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/Resource EKSClusterDefaultVpcPublicSubnet1Subnet516BB131: Type: AWS::EC2::Subnet Properties: AvailabilityZone: Fn::Select: - 0 - Fn::GetAZs: "" CidrBlock: 10.0.0.0/18 MapPublicIpOnLaunch: true Tags: - Key: aws-cdk:subnet-name Value: Public - Key: aws-cdk:subnet-type Value: Public - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/Subnet EKSClusterDefaultVpcPublicSubnet1RouteTable64518699: Type: AWS::EC2::RouteTable Properties: Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/RouteTable EKSClusterDefaultVpcPublicSubnet1RouteTableAssociationBC31FBB4: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: Ref: EKSClusterDefaultVpcPublicSubnet1RouteTable64518699 SubnetId: Ref: EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/RouteTableAssociation EKSClusterDefaultVpcPublicSubnet1DefaultRoute36D7F29D: Type: AWS::EC2::Route Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: Ref: EKSClusterDefaultVpcIGWD0D5B12A RouteTableId: Ref: EKSClusterDefaultVpcPublicSubnet1RouteTable64518699 DependsOn: - EKSClusterDefaultVpcVPCGW47C2C18C Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/DefaultRoute EKSClusterDefaultVpcPublicSubnet1EIPD07F43CC: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/EIP EKSClusterDefaultVpcPublicSubnet1NATGateway209E5D96: Type: AWS::EC2::NatGateway Properties: AllocationId: Fn::GetAtt: - EKSClusterDefaultVpcPublicSubnet1EIPD07F43CC - AllocationId SubnetId: Ref: EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1 DependsOn: - EKSClusterDefaultVpcPublicSubnet1DefaultRoute36D7F29D - EKSClusterDefaultVpcPublicSubnet1RouteTableAssociationBC31FBB4 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet1/NATGateway EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E: Type: AWS::EC2::Subnet Properties: AvailabilityZone: Fn::Select: - 1 - Fn::GetAZs: "" CidrBlock: 10.0.64.0/18 MapPublicIpOnLaunch: true Tags: - Key: aws-cdk:subnet-name Value: Public - Key: aws-cdk:subnet-type Value: Public - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/Subnet EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726: Type: AWS::EC2::RouteTable Properties: Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/RouteTable EKSClusterDefaultVpcPublicSubnet2RouteTableAssociation668702D0: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: Ref: EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726 SubnetId: Ref: EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/RouteTableAssociation EKSClusterDefaultVpcPublicSubnet2DefaultRoute36B802DF: Type: AWS::EC2::Route Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: Ref: EKSClusterDefaultVpcIGWD0D5B12A RouteTableId: Ref: EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726 DependsOn: - EKSClusterDefaultVpcVPCGW47C2C18C Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/DefaultRoute EKSClusterDefaultVpcPublicSubnet2EIPDE6AFFB2: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/EIP EKSClusterDefaultVpcPublicSubnet2NATGateway7779FD2A: Type: AWS::EC2::NatGateway Properties: AllocationId: Fn::GetAtt: - EKSClusterDefaultVpcPublicSubnet2EIPDE6AFFB2 - AllocationId SubnetId: Ref: EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E Tags: - Key: kubernetes.io/role/elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2 DependsOn: - EKSClusterDefaultVpcPublicSubnet2DefaultRoute36B802DF - EKSClusterDefaultVpcPublicSubnet2RouteTableAssociation668702D0 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PublicSubnet2/NATGateway EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89: Type: AWS::EC2::Subnet Properties: AvailabilityZone: Fn::Select: - 0 - Fn::GetAZs: "" CidrBlock: 10.0.128.0/18 MapPublicIpOnLaunch: false Tags: - Key: aws-cdk:subnet-name Value: Private - Key: aws-cdk:subnet-type Value: Private - Key: kubernetes.io/role/internal-elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1/Subnet EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2: Type: AWS::EC2::RouteTable Properties: Tags: - Key: kubernetes.io/role/internal-elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1/RouteTable EKSClusterDefaultVpcPrivateSubnet1RouteTableAssociationCA4DCA5F: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: Ref: EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2 SubnetId: Ref: EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1/RouteTableAssociation EKSClusterDefaultVpcPrivateSubnet1DefaultRoute658B5596: Type: AWS::EC2::Route Properties: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: Ref: EKSClusterDefaultVpcPublicSubnet1NATGateway209E5D96 RouteTableId: Ref: EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet1/DefaultRoute EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B: Type: AWS::EC2::Subnet Properties: AvailabilityZone: Fn::Select: - 1 - Fn::GetAZs: "" CidrBlock: 10.0.192.0/18 MapPublicIpOnLaunch: false Tags: - Key: aws-cdk:subnet-name Value: Private - Key: aws-cdk:subnet-type Value: Private - Key: kubernetes.io/role/internal-elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2/Subnet EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D: Type: AWS::EC2::RouteTable Properties: Tags: - Key: kubernetes.io/role/internal-elb Value: "1" - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2 VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2/RouteTable EKSClusterDefaultVpcPrivateSubnet2RouteTableAssociation3202B241: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: Ref: EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D SubnetId: Ref: EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2/RouteTableAssociation EKSClusterDefaultVpcPrivateSubnet2DefaultRouteDE9B9A5A: Type: AWS::EC2::Route Properties: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: Ref: EKSClusterDefaultVpcPublicSubnet2NATGateway7779FD2A RouteTableId: Ref: EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/PrivateSubnet2/DefaultRoute EKSClusterDefaultVpcIGWD0D5B12A: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: NormalStackForEksProduct/EKSCluster/DefaultVpc Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/IGW EKSClusterDefaultVpcVPCGW47C2C18C: Type: AWS::EC2::VPCGatewayAttachment Properties: InternetGatewayId: Ref: EKSClusterDefaultVpcIGWD0D5B12A VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/DefaultVpc/VPCGW EKSClusterKubectlHandlerRoleCB45095B: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: "2012-10-17" ManagedPolicyArns: - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - :iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - :iam::aws:policy/AmazonEC2ContainerRegistryReadOnly - Fn::If: - EKSClusterHasEcrPublic0E1E292B - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - :iam::aws:policy/AmazonElasticContainerRegistryPublicReadOnly - Ref: AWS::NoValue Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/KubectlHandlerRole/Resource EKSClusterKubectlHandlerRoleDefaultPolicyFF7EEBDC: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: eks:DescribeCluster Effect: Allow Resource: Fn::GetAtt: - EKSClusterE11008B6 - Arn - Action: sts:AssumeRole Effect: Allow Resource: Fn::GetAtt: - EKSClusterCreationRoleB865C9E8 - Arn Version: "2012-10-17" PolicyName: EKSClusterKubectlHandlerRoleDefaultPolicyFF7EEBDC Roles: - Ref: EKSClusterKubectlHandlerRoleCB45095B Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/KubectlHandlerRole/DefaultPolicy/Resource EKSClusterRoleC0AEAC3D: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: eks.amazonaws.com Version: "2012-10-17" ManagedPolicyArns: - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - :iam::aws:policy/AmazonEKSClusterPolicy Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/Role/Resource EKSClusterControlPlaneSecurityGroup580AD1FE: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: EKS Control Plane Security Group SecurityGroupEgress: - CidrIp: 0.0.0.0/0 Description: Allow all outbound traffic by default IpProtocol: "-1" VpcId: Ref: EKSClusterDefaultVpc85D27B27 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/ControlPlaneSecurityGroup/Resource EKSClusterCreationRoleB865C9E8: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: AWS: Fn::GetAtt: - awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454 - Outputs.NormalStackForEksProductawscdkawseksClusterResourceProviderOnEventHandlerServiceRole3C3F4AB3Arn - Action: sts:AssumeRole Effect: Allow Principal: AWS: Fn::GetAtt: - awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454 - Outputs.NormalStackForEksProductawscdkawseksClusterResourceProviderIsCompleteHandlerServiceRoleF1E75521Arn - Action: sts:AssumeRole Effect: Allow Principal: AWS: Fn::GetAtt: - EKSClusterKubectlHandlerRoleCB45095B - Arn Version: "2012-10-17" DependsOn: - EKSClusterDefaultVpcIGWD0D5B12A - EKSClusterDefaultVpcPrivateSubnet1DefaultRoute658B5596 - EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2 - EKSClusterDefaultVpcPrivateSubnet1RouteTableAssociationCA4DCA5F - EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 - EKSClusterDefaultVpcPrivateSubnet2DefaultRouteDE9B9A5A - EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D - EKSClusterDefaultVpcPrivateSubnet2RouteTableAssociation3202B241 - EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B - EKSClusterDefaultVpcPublicSubnet1DefaultRoute36D7F29D - EKSClusterDefaultVpcPublicSubnet1EIPD07F43CC - EKSClusterDefaultVpcPublicSubnet1NATGateway209E5D96 - EKSClusterDefaultVpcPublicSubnet1RouteTable64518699 - EKSClusterDefaultVpcPublicSubnet1RouteTableAssociationBC31FBB4 - EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 - EKSClusterDefaultVpcPublicSubnet2DefaultRoute36B802DF - EKSClusterDefaultVpcPublicSubnet2EIPDE6AFFB2 - EKSClusterDefaultVpcPublicSubnet2NATGateway7779FD2A - EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726 - EKSClusterDefaultVpcPublicSubnet2RouteTableAssociation668702D0 - EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E - EKSClusterDefaultVpc85D27B27 - EKSClusterDefaultVpcVPCGW47C2C18C Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/Resource/CreationRole/Resource EKSClusterCreationRoleDefaultPolicy27A5F6BE: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: iam:PassRole Effect: Allow Resource: Fn::GetAtt: - EKSClusterRoleC0AEAC3D - Arn - Action: - eks:CreateCluster - eks:DescribeCluster - eks:DescribeUpdate - eks:DeleteCluster - eks:UpdateClusterVersion - eks:UpdateClusterConfig - eks:CreateFargateProfile - eks:TagResource - eks:UntagResource Effect: Allow Resource: - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - ":eks:" - Ref: AWS::Region - ":" - Ref: AWS::AccountId - :cluster/MySimpleEKSCluster - Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - ":eks:" - Ref: AWS::Region - ":" - Ref: AWS::AccountId - :cluster/MySimpleEKSCluster/* - Action: - eks:DescribeFargateProfile - eks:DeleteFargateProfile Effect: Allow Resource: Fn::Join: - "" - - "arn:" - Ref: AWS::Partition - ":eks:" - Ref: AWS::Region - ":" - Ref: AWS::AccountId - :fargateprofile/MySimpleEKSCluster/* - Action: - iam:GetRole - iam:listAttachedRolePolicies Effect: Allow Resource: "*" - Action: iam:CreateServiceLinkedRole Effect: Allow Resource: "*" - Action: - ec2:DescribeInstances - ec2:DescribeNetworkInterfaces - ec2:DescribeSecurityGroups - ec2:DescribeSubnets - ec2:DescribeRouteTables - ec2:DescribeDhcpOptions - ec2:DescribeVpcs Effect: Allow Resource: "*" Version: "2012-10-17" PolicyName: EKSClusterCreationRoleDefaultPolicy27A5F6BE Roles: - Ref: EKSClusterCreationRoleB865C9E8 DependsOn: - EKSClusterDefaultVpcIGWD0D5B12A - EKSClusterDefaultVpcPrivateSubnet1DefaultRoute658B5596 - EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2 - EKSClusterDefaultVpcPrivateSubnet1RouteTableAssociationCA4DCA5F - EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 - EKSClusterDefaultVpcPrivateSubnet2DefaultRouteDE9B9A5A - EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D - EKSClusterDefaultVpcPrivateSubnet2RouteTableAssociation3202B241 - EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B - EKSClusterDefaultVpcPublicSubnet1DefaultRoute36D7F29D - EKSClusterDefaultVpcPublicSubnet1EIPD07F43CC - EKSClusterDefaultVpcPublicSubnet1NATGateway209E5D96 - EKSClusterDefaultVpcPublicSubnet1RouteTable64518699 - EKSClusterDefaultVpcPublicSubnet1RouteTableAssociationBC31FBB4 - EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 - EKSClusterDefaultVpcPublicSubnet2DefaultRoute36B802DF - EKSClusterDefaultVpcPublicSubnet2EIPDE6AFFB2 - EKSClusterDefaultVpcPublicSubnet2NATGateway7779FD2A - EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726 - EKSClusterDefaultVpcPublicSubnet2RouteTableAssociation668702D0 - EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E - EKSClusterDefaultVpc85D27B27 - EKSClusterDefaultVpcVPCGW47C2C18C Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/Resource/CreationRole/DefaultPolicy/Resource EKSClusterE11008B6: Type: Custom::AWSCDK-EKS-Cluster Properties: ServiceToken: Fn::GetAtt: - awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454 - Outputs.NormalStackForEksProductawscdkawseksClusterResourceProviderframeworkonEvent992B4ED3Arn Config: name: MySimpleEKSCluster version: "1.25" roleArn: Fn::GetAtt: - EKSClusterRoleC0AEAC3D - Arn kubernetesNetworkConfig: ipFamily: ipv4 resourcesVpcConfig: subnetIds: - Ref: EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 - Ref: EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E - Ref: EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 - Ref: EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B securityGroupIds: - Fn::GetAtt: - EKSClusterControlPlaneSecurityGroup580AD1FE - GroupId endpointPublicAccess: true endpointPrivateAccess: true AssumeRoleArn: Fn::GetAtt: - EKSClusterCreationRoleB865C9E8 - Arn AttributesRevision: 2 DependsOn: - EKSClusterDefaultVpcIGWD0D5B12A - EKSClusterDefaultVpcPrivateSubnet1DefaultRoute658B5596 - EKSClusterDefaultVpcPrivateSubnet1RouteTable1A182CC2 - EKSClusterDefaultVpcPrivateSubnet1RouteTableAssociationCA4DCA5F - EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 - EKSClusterDefaultVpcPrivateSubnet2DefaultRouteDE9B9A5A - EKSClusterDefaultVpcPrivateSubnet2RouteTable735AFA1D - EKSClusterDefaultVpcPrivateSubnet2RouteTableAssociation3202B241 - EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B - EKSClusterDefaultVpcPublicSubnet1DefaultRoute36D7F29D - EKSClusterDefaultVpcPublicSubnet1EIPD07F43CC - EKSClusterDefaultVpcPublicSubnet1NATGateway209E5D96 - EKSClusterDefaultVpcPublicSubnet1RouteTable64518699 - EKSClusterDefaultVpcPublicSubnet1RouteTableAssociationBC31FBB4 - EKSClusterDefaultVpcPublicSubnet1Subnet516BB131 - EKSClusterDefaultVpcPublicSubnet2DefaultRoute36B802DF - EKSClusterDefaultVpcPublicSubnet2EIPDE6AFFB2 - EKSClusterDefaultVpcPublicSubnet2NATGateway7779FD2A - EKSClusterDefaultVpcPublicSubnet2RouteTable29E9C726 - EKSClusterDefaultVpcPublicSubnet2RouteTableAssociation668702D0 - EKSClusterDefaultVpcPublicSubnet2Subnet5E7B0B3E - EKSClusterDefaultVpc85D27B27 - EKSClusterDefaultVpcVPCGW47C2C18C - EKSClusterCreationRoleDefaultPolicy27A5F6BE - EKSClusterCreationRoleB865C9E8 UpdateReplacePolicy: Delete DeletionPolicy: Delete Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/Resource/Resource/Default EKSClusterKubectlReadyBarrierEF5C3970: Type: AWS::SSM::Parameter Properties: Type: String Value: aws:cdk:eks:kubectl-ready DependsOn: - EKSClusterCreationRoleDefaultPolicy27A5F6BE - EKSClusterCreationRoleB865C9E8 - EKSClusterE11008B6 Metadata: aws:cdk:path: NormalStackForEksProduct/EKSCluster/KubectlReadyBarrier awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454: Type: AWS::CloudFormation::Stack Properties: TemplateURL: Fn::Join: - "" - - https://s3. - Ref: AWS::Region - "." - Ref: AWS::URLSuffix - / - Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region} - /a06289691ca9860950e42351ce423e5d68eb00af41fd68c93ad3151034dce88b.json UpdateReplacePolicy: Delete DeletionPolicy: Delete Metadata: aws:cdk:path: NormalStackForEksProduct/@aws-cdk--aws-eks.ClusterResourceProvider.NestedStack/@aws-cdk--aws-eks.ClusterResourceProvider.NestedStackResource aws:asset:path: NormalStackForEksProductawscdkawseksClusterResourceProviderB5F07F5C.nested.template.json aws:asset:property: TemplateURL awscdkawseksKubectlProviderNestedStackawscdkawseksKubectlProviderNestedStackResourceA7AEBA6B: Type: AWS::CloudFormation::Stack Properties: Parameters: referencetoNormalStackForEksProductEKSClusterKubectlHandlerRole05CE6321Arn: Fn::GetAtt: - EKSClusterKubectlHandlerRoleCB45095B - Arn referencetoNormalStackForEksProductEKSClusterDefaultVpcPrivateSubnet1SubnetE7BA1646Ref: Ref: EKSClusterDefaultVpcPrivateSubnet1Subnet90762A89 referencetoNormalStackForEksProductEKSClusterDefaultVpcPrivateSubnet2SubnetB066EA98Ref: Ref: EKSClusterDefaultVpcPrivateSubnet2SubnetD005851B referencetoNormalStackForEksProductEKSCluster536E68B6ClusterSecurityGroupId: Fn::GetAtt: - EKSClusterE11008B6 - ClusterSecurityGroupId TemplateURL: Fn::Join: - "" - - https://s3. - Ref: AWS::Region - "." - Ref: AWS::URLSuffix - / - Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region} - /e4e47ee52536b9251914400ada9eccd5ef5c9adb4e89561d298288e1ed328059.json DependsOn: - EKSClusterDefaultVpcPrivateSubnet1DefaultRoute658B5596 - EKSClusterDefaultVpcPrivateSubnet1RouteTableAssociationCA4DCA5F - EKSClusterDefaultVpcPrivateSubnet2DefaultRouteDE9B9A5A - EKSClusterDefaultVpcPrivateSubnet2RouteTableAssociation3202B241 - EKSClusterKubectlHandlerRoleDefaultPolicyFF7EEBDC - EKSClusterKubectlHandlerRoleCB45095B UpdateReplacePolicy: Delete DeletionPolicy: Delete Metadata: aws:cdk:path: NormalStackForEksProduct/@aws-cdk--aws-eks.KubectlProvider.NestedStack/@aws-cdk--aws-eks.KubectlProvider.NestedStackResource aws:asset:path: NormalStackForEksProductawscdkawseksKubectlProviderB5D778AC.nested.template.json aws:asset:property: TemplateURL CDKMetadata: Type: AWS::CDK::Metadata Properties: Analytics: v2:deflate64:H4sIAAAAAAAA/02QQYvCMBCFf4v3NLvrwrIepSziZQmteJU0jjjbNiPJRCml/920XY2n+ebxMryXpVx9y/eFvvnMHOuswUr2JWtTiygdeqi9zJvgGZwAs5T9/mJEfrJ7lQsVqgZNGSoLPGqJCgoMO101kPSkrb0ng5qR7NM8ws9WjeNX80Yz3HQnlMNrxHR4a2OQyA/DnOR/W3PMfW7BsijBBIfcbRyFy5ThVRgE6lb2Bc35pqkodpkOzjQMogBPwZnoif2pTWt88sI52SM+ykxfNwjV8Zns26dcyY+vxZ9HzFywjC3IYp53IJyFcHgBAAA= Metadata: aws:cdk:path: NormalStackForEksProduct/CDKMetadata/Default Condition: CDKMetadataAvailable Conditions: EKSClusterHasEcrPublic0E1E292B: Fn::Equals: - Ref: AWS::Partition - aws CDKMetadataAvailable: Fn::Or: - Fn::Or: - Fn::Equals: - Ref: AWS::Region - af-south-1 - Fn::Equals: - Ref: AWS::Region - ap-east-1 - Fn::Equals: - Ref: AWS::Region - ap-northeast-1 - Fn::Equals: - Ref: AWS::Region - ap-northeast-2 - Fn::Equals: - Ref: AWS::Region - ap-south-1 - Fn::Equals: - Ref: AWS::Region - ap-southeast-1 - Fn::Equals: - Ref: AWS::Region - ap-southeast-2 - Fn::Equals: - Ref: AWS::Region - ca-central-1 - Fn::Equals: - Ref: AWS::Region - cn-north-1 - Fn::Equals: - Ref: AWS::Region - cn-northwest-1 - Fn::Or: - Fn::Equals: - Ref: AWS::Region - eu-central-1 - Fn::Equals: - Ref: AWS::Region - eu-north-1 - Fn::Equals: - Ref: AWS::Region - eu-south-1 - Fn::Equals: - Ref: AWS::Region - eu-west-1 - Fn::Equals: - Ref: AWS::Region - eu-west-2 - Fn::Equals: - Ref: AWS::Region - eu-west-3 - Fn::Equals: - Ref: AWS::Region - me-south-1 - Fn::Equals: - Ref: AWS::Region - sa-east-1 - Fn::Equals: - Ref: AWS::Region - us-east-1 - Fn::Equals: - Ref: AWS::Region - us-east-2 - Fn::Or: - Fn::Equals: - Ref: AWS::Region - us-west-1 - Fn::Equals: - Ref: AWS::Region - us-west-2 Parameters: BootstrapVersion: Type: AWS::SSM::Parameter::Value Default: /cdk-bootstrap/hnb659fds/version Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip] Rules: CheckBootstrapVersion: Assertions: - Assert: Fn::Not: - Fn::Contains: - - "1" - "2" - "3" - "4" - "5" - Ref: BootstrapVersion AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI. ```
Second not working code (but we want to have this in that way): ```python import aws_cdk as cdk from aws_cdk import ( aws_eks as eks, aws_servicecatalog as sc, App, ) from constructs import Construct class EKSProduct(sc.ProductStack): def __init__(self, scope: Construct, id: str) -> None: super().__init__(scope, id) cluster = eks.Cluster(self, 'EKSCluster', default_capacity=0, version=eks.KubernetesVersion.V1_25, cluster_name="MySimpleEKSCluster", ) class WrapperStack(cdk.Stack): def __init__(self, scope: Construct, id: str) -> None: super().__init__(scope, id) EKSProduct(self, "EksProductInsideWrapper") app = App() WrapperStack(app, "WrapperStackForEksProduct") app.synth() ```
Response of cdk synth: ```console jsii.errors.JavaScriptError: Error: An Asset Bucket must be provided to use Assets at ProductStackSynthesizer.addFileAsset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-servicecatalog/lib/private/product-stack-synthesizer.js:1:884) at NestedStackSynthesizer.addFileAsset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/core/lib/stack-synthesizers/nested.js:1:1025) at new Asset (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1207) at AssetCode.bind (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4628) at new LayerVersion (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-lambda/lib/layers.js:1:2066) at new NodeProxyAgentLayer (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/lambda-layer-node-proxy-agent/lib/node-proxy-agent-layer.js:1:374) at new ClusterResourceProvider (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.js:1:698) at ClusterResourceProvider.getOrCreate (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.js:1:583) at new ClusterResource (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-eks/lib/cluster-resource.js:1:572) at new Cluster (/var/folders/_j/mjh9f5w13p11vvtstsqpv8hr0000gq/T/jsii-kernel-hL8KQR/node_modules/aws-cdk-lib/aws-eks/lib/cluster.js:1:11473) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/xxx/ECAS/service-catalog/app_eks.py", line 29, in WrapperStack(app, "WrapperStackForEksProduct") File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__ inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs) File "/Users/xxx/ECAS/service-catalog/app_eks.py", line 25, in __init__ EKSProduct(self, "EksProductInsideWrapper") File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__ inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs) File "/Users/xxx/ECAS/service-catalog/app_eks.py", line 15, in __init__ cluster = eks.Cluster(self, 'EKSCluster', File "/Users/xxx/lib/python3.9/site-packages/jsii/_runtime.py", line 118, in __call__ inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs) File "/Users/xxx/lib/python3.9/site-packages/aws_cdk/aws_eks/__init__.py", line 13846, in __init__ jsii.create(self.__class__, self, [scope, id, props]) File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/__init__.py", line 334, in create response = self.provider.create( File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/providers/process.py", line 365, in create return self._process.send(request, CreateResponse) File "/Users/xxx/lib/python3.9/site-packages/jsii/_kernel/providers/process.py", line 342, in send raise RuntimeError(resp.error) from JavaScriptError(resp.stack) RuntimeError: An Asset Bucket must be provided to use Assets ```

So we do not know how to provide bucket to assets and why default bucket (that is in first working code, I think create by AWS/CDK) is not pass to ProductStack

peterwoodworth commented 1 year ago

The first stack will succeed because it will deploy the assets to the bucket that you've bootstrapped with. ProductStacks are distinct in that you need to specifically name the asset bucket to be able to use assets. You can specify the bucket as an input prop to the ProductStack construct when you instantiate it

import aws_cdk as cdk
from aws_cdk import (
    aws_eks as eks,
    aws_servicecatalog as sc,
    aws_s3 as s3,
    App,
)

from constructs import Construct

class EKSProduct(sc.ProductStack):
    def __init__(self, scope: Construct, id: str, asset_bucket: s3.Bucket) -> None:
        super().__init__(scope, id, asset_bucket=asset_bucket)

        cluster = eks.Cluster(self, 'EKSCluster',
                              default_capacity=0,
                              version=eks.KubernetesVersion.V1_25,
                              cluster_name="MySimpleEKSCluster",
                              )

class WrapperStack(cdk.Stack):
    def __init__(self, scope: Construct, id: str) -> None:
        super().__init__(scope, id)

        bucket = s3.Bucket(self, "Bucket", bucket_name="mybucket")

        EKSProduct(self, "EksProductInsideWrapper", asset_bucket=bucket)

app = App()
WrapperStack(app, "WrapperStackForEksProduct")
app.synth()
github-actions[bot] commented 1 year ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.