aws-quickstart / cdk-eks-blueprints

AWS Quick Start Team
Apache License 2.0
460 stars 207 forks source link

ApplicationTeam: Deployment fails due to CFN output dependency when application team is removed #596

Closed adamjkeller closed 1 year ago

adamjkeller commented 1 year ago

Describe the bug

When removing an application team from the cdk app, the cdk deployment will fail with the following error:

Export ekscdkbp:ExportsOutputFnGetAttNSRoletestTeam252AE2E51Arn4085A00C cannot be deleted as it is in use by ekscdkbp-eks

This is due to a CFN export being created in one stack and relying on it in another. The workaround to this is to tell the cdk to deploy the dependent stack first, and then the other stack that outputs the Role ARN.

Workaround example:

  npm run cdk -- deploy --exclusively "ekscdkbp-eks" 
  npm run cdk -- deploy --exclusively "ekscdkbp-cluster" 

This is less than ideal as it requires the deployment automation to require custom logic to deploy in that particular order only when an application team is removed.

Expected Behavior

I should be able to run a cdk deploy --all without having to work around the dependency management from CFN exports.

Current Behavior

Described in first section

Reproduction Steps

https://github.com/aws-containers/proton-codebuild-provisioning-examples/blob/3273f247e010041603e7d16fbf75ed3de867d520/cdk/environment-templates/cdk-vpc-eks-cluster/v1/infrastructure/lib/vpc-eks-cluster-stack.ts#L62

https://github.com/aws-containers/proton-codebuild-provisioning-examples/blob/eks-example/cdk/environment-templates/cdk-vpc-eks-cluster/v1/infrastructure/lib/teams.ts

https://github.com/aws-containers/proton-codebuild-provisioning-examples/blob/3273f247e010041603e7d16fbf75ed3de867d520/cdk/environment-templates/cdk-vpc-eks-cluster/v1/infrastructure/manifest.yaml#L11

Possible Solution

Avoid using CFN Exports and use SSM parameter store. Or move the dependency into the same stack.

Additional Information/Context

No response

CDK CLI Version

2.60.0

EKS Blueprints Version

1.5.4

Node.js Version

v16.17.0

Environment details (OS name and version, etc.)

Mac OS

Other information

No response

shapirov103 commented 1 year ago

@adamjkeller I looked at the code and I would like to understand your use case better. Why do you need the outer stack in the first place?

If it is for VPC injection (or any other resources), you can wrap it with a resource provider.

You can create a class like MyVpcProvider implements ResourceProvider<IVpc>. Simple logic can be expressed like this:

const app = new cdk.App();
...

blueprints.EksBlueprint.builder()
            .addOns(...addOns)
            .clusterProvider(clusterProvider)
            .resourceProvider(GlobalResources.Vpc, {
                provide(context: blueprints.ResourceContext) : IVpc {
                    return new ec2.Vpc(context.scope, "my-vpc");
                }
            })
           ...
           .build(scope, blueprintID, props);

Outer class can be retained if you would like any parameter logic, but perhaps it does not have to extend stack. We tested pushing multiple teams in a complex enterprise with pipelines support.

adamjkeller commented 1 year ago

Thanks for the response @shapirov103! I definitely don't need the additional stack and didn't realize that by creating the VPC within the app would cause this to happen. I am building out a demo for proton to deploy EKS clusters with the CDK and EKS blueprints and ran into this issue.

I will try to use resource providers, which it sounds like will help with the dependency issues with CFN exports. I will test this tomorrow and update. Thanks again!