MWAAEnvironment fails to deploy as part of CICDPipelineStack

[devansh-gandhi]

Description:

cdk synth/CodeBuild step fails when deploying MWAAEnvironment through CICDPipeline.

Steps to Reproduce:

Add MWAAEnvironemnt to a stage of the ddk CICDPipelineStack
updated "cdk synth" command in the codeBuild step to "cdk synth --verbose" to generate detailed logs

Expected Behavior: Cdk synth runs successfully and the MWAAEnvironment gets deployed

Actual Behavior: cdk synth in CodeBuild step of CICD pipeline fails. The AZ information is missing from the cdk context (cdk.json /cdk.context.json). i believe this information is required to create the VPC. Therefore the codebuild iam role tries to assume the cdk-lookup role but fails as it does not have the permissions to assume that role. This results in the entire cdk synth/codeBuild to fail.

Some of the error messages from the logs -

[17:26:55] Some context information is missing. Fetching... [17:26:55] Retrieved account ID {account_id} from disk cache [17:26:55] Reading AZs for {account_id}:us-east-1 [17:26:55] Assuming role 'arn:aws:iam::{account_id}:role/cdk-hnb659fds-lookup-role-{account_id}-us-east-1'. [17:26:55] Assuming role failed: User: arn:aws:sts::{account_id}:assumed-role/DDKCodePipelineBuildSynthCdkBuil-{hash}/AWSCodeBuild-{hash} is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::{account_id}:role/cdk-hnb659fds-lookup-role-{account_id}-us-east-1 [17:26:55] Could not assume role in target account using current credentials User: arn:aws:sts::{account_id}:assumed-role/{repo}-DDKCodePipelineBuildSynthCdkBuil-{hash}/AWSCodeBuild-{hash} is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::{account_id}:role/cdk-hnb659fds-lookup-role-{account_id}-us-east-1 . Please make sure that this role exists in the account. If it doesn't exist, (re)-bootstrap the environment with the right '--trust', using the latest version of the CDK CLI. current credentials could not be used to assume 'arn:aws:iam::{account_id}:role/cdk-hnb659fds-lookup-role-{account_id}-us-east-1', but are for the right account. Proceeding anyway. [17:26:55] Call failed: describeAvailabilityZones(undefined) => You are not authorized to perform this operation. (code=UnauthorizedOperation) [17:26:55] Setting "availability-zones:account={account_id}:region=us-east-1" context to {"$providerError":"You are not authorized to perform this operation.","$dontSaveContext":true}

Environment:

Operating System:

Browser (if applicable):

Programming Language: Python3 Framework/Library (if applicable): requirements.txt

    aws_ddk_core==1.1.0
    boto3==1.24
    aws-cdk-lib==2.85.0
    aws-cdk.aws-glue-alpha==2.85.0a0
    aws-cdk.aws-kinesisfirehose-alpha==2.85.0a0
    aws-cdk.aws-kinesisfirehose-destinations-alpha==2.85.0a0
    constructs>=10.0.5

Other relevant information:

https://docs.aws.amazon.com/cdk/v2/guide/context.html

example of cdk app where the az is passed - https://github.com/aws-samples/designing-cloud-native-microservices-on-aws/blob/63d254b09efd442d5f83fcfd81853f611eb5c8ab/deployment/coffeeshop-cdk/cdk.context.json

CDK issue with possible solution - https://github.com/aws/aws-cdk/issues/20975

[malachi-constant]

Thanks for opening this, i'll take a look shortly.

[malachi-constant]

This works with no issues in the typescript version

import * as cdk from "aws-cdk-lib";
import * as ddk from "aws-ddk-core";
import { Construct } from "constructs";

const app = new cdk.App();

class ApplicationStage extends cdk.Stage {
  constructor(scope: Construct, id: string,kwargs = {}) {
      super(scope, "MWAAEnvironmentStage");
      const stack = new cdk.Stack(this, "MWAAEnvironment")
      new ddk.MWAAEnvironment(stack, "Airflow", 
        {
          name: "TestAirflowEnv",
          vpcCidr: "10.56.0.0/16"
        }
      );
  }
}

new ddk.CICDPipelineStack(app, "DdkCICDPipeline", {
  pipelineName: "ddk-cicd-pipeline",
  cdkLanguage: "typescript",
})
  .addSourceAction({
    repositoryName: "ddk-mwaa-stage-test",
  })
  .addSynthAction({})
  .buildPipeline()
  .addStage({
    stageId: "dev",
    stage: new ApplicationStage(app, "dev"),
  })
  .synth();

Trying to replicate in python now

[malachi-constant]

Python version works as well for me.

aws_ddk_core=1.1.0 aws_cdk_lib=2.85.0

import aws_cdk as cdk
import aws_ddk_core as ddk
from constructs import Construct
from typing import Any

app = cdk.App()

class ApplicationStage(cdk.Stage):
    def __init__(self, scope: Construct, id: str, **kwargs: Any) -> None:
      super().__init__(scope, id, **kwargs)
      stack = cdk.Stack(self, "MWAAEnvironment")
      ddk.MWAAEnvironment(stack, "Airflow",
          name="TestAirflowEnv",
          vpc_cidr="10.56.0.0/16"
      )

pipeline = (
    ddk.CICDPipelineStack(
        app,
        id="ddk-cicd-pipeline",
        pipeline_name="ddk-cicd-pipeline",
        cdk_language="python",
    )
    .add_source_action(repository_name="ddk-mwaa-stage-test")
    .add_synth_action()
    .build_pipeline()
    .add_stage(stage_id="dev", stage=ApplicationStage(app, "dev"))
    .synth()
)

app.synth()

Can you provide me any more code to try and replicate your issue @devansh-gandhi ? Also are you checking in cdk.context.json to codecommit as well?

[devansh-gandhi]

No, I did not commit the cdk.context.json file. I think I did not run cdk synth locally after adding the MWAAEnvironment, therefore the cdk.context.json file did not get created and committed. I will commit the cdk.context.json file and test it out. Thank you!

[malachi-constant]

Ok, let me know if anything changes, fyi I did not include any context file and just ran it as pasted above.

[devansh-gandhi]

Hi, I successfully deployed the example you provided; however, I got an error when attempting to deploy it via the SDLF lightweight example without the cdk.context.json file. Therefore, I can open an issue on the aws-ddk-examples instead. We can close this issue. Thank you!

[anmolsgandhi]

Thanks @devansh-gandhi Feel free to open an issue there and we will try to get to it!