aws-codepipeline: CodePipeline/CodeBuild roles don't have necessary access

[jesseadams]

Describe the bug

I am trying to deploy a CDK stack through CodePipeline. I am defining the CodePipeline configuration to automate this as a separate CDK stack. I've tried two different iterations and both of them result in some kind of IAM authorization issue.

Iteration 1:

The pipeline stack deploys, but the CodeBuild build job fails.

Iteration 2:

I tried to create a custom IAM role with admin access and pass it in to CodeBuild but the CodePipeline stack no longer will deploy.

Expected Behavior

I would have expected either iteration to have successfully deployed the CodePipeline stack and deployed the the other stack in CodeBuild.

Current Behavior

Neither iterations work.

Error with iteration 1:

Deployment failed: Error: example-stack: This CDK deployment requires bootstrap stack version '6', but during the confirmation via SSM parameter /cdk-bootstrap/hnb659fds/version the following error occurred: AccessDeniedException: User: arn:aws:sts::OMITTED:assumed-role/example-testRole34633740-AyqzAsAGDOPz/AWSCodeBuild-b20ef2ed-7da6-49f2-8afd-3604ae3ddc09 is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-east-2:OMITTED:parameter/cdk-bootstrap/hnb659fds/version because no identity-based policy allows the ssm:GetParameter action

Error with iteration 2:

arn:aws:iam::OMITTED:role/example-automationRole287FA533-Vtj1n51Shw46 is not authorized to perform AssumeRole on role arn:aws:iam::OMITTED:role/example-CodeBuildRole728CBADE-z8z8uvmDfzC (Service: AWSCodePipeline; Status Code: 400; Error Code: InvalidStructureException; Request ID: 176eb989-f485
-49c6-8fe7-d816bf5c6018; Proxy: null)

Reproduction Steps

Iteration 1:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as codepipeline from 'aws-cdk-lib/aws-codepipeline';
import * as codepipeline_actions from 'aws-cdk-lib/aws-codepipeline-actions';
import * as codebuild from 'aws-cdk-lib/aws-codebuild';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';
import { SecretValue } from 'aws-cdk-lib';

export class PipelineStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const githubTokenString = process.env.GITHUB_TOKEN || '';
    const githubToken = new Secret(this, 'github-token', {
      secretStringValue: SecretValue.unsafePlainText(githubTokenString),
    });

    const pipeline = new codepipeline.Pipeline(this, 'automation', {});
    const sourceOutput = new codepipeline.Artifact();

    const sourceAction = new codepipeline_actions.GitHubSourceAction({
      actionName: 'Github',
      owner: example',
      repo: 'example',
      oauthToken: githubToken.secretValue,
      output: sourceOutput,
      branch: 'main',
    });

    const buildAction = new codepipeline_actions.CodeBuildAction({
      actionName: 'Build',
      project: new codebuild.PipelineProject(this, 'build', {
        buildSpec: codebuild.BuildSpec.fromSourceFilename('buildspec.yaml'),
        environment: {
          buildImage: codebuild.LinuxBuildImage.STANDARD_7_0,
        },
      }),
      input: sourceOutput,
    });

    pipeline.addStage({
      stageName: 'Source',
      actions: [sourceAction],
    });

    pipeline.addStage({
      stageName: 'Build',
      actions: [buildAction],
    });

  }
}

Iteration 2:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as codepipeline from 'aws-cdk-lib/aws-codepipeline';
import * as codepipeline_actions from 'aws-cdk-lib/aws-codepipeline-actions';
import * as codebuild from 'aws-cdk-lib/aws-codebuild';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';
import { SecretValue } from 'aws-cdk-lib';

export class PipelineStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const githubTokenString = process.env.GITHUB_TOKEN || '';
    const githubToken = new Secret(this, 'github-token', {
      secretStringValue: SecretValue.unsafePlainText(githubTokenString),
    });

    const pipeline = new codepipeline.Pipeline(this, 'automation', {});
    const sourceOutput = new codepipeline.Artifact();

    const sourceAction = new codepipeline_actions.GitHubSourceAction({
      actionName: 'Github',
      owner: example',
      repo: 'example',
      oauthToken: githubToken.secretValue,
      output: sourceOutput,
      branch: 'main',
    });

    const buildAction = new codepipeline_actions.CodeBuildAction({
      actionName: 'Build',
      project: new codebuild.PipelineProject(this, 'build', {
        buildSpec: codebuild.BuildSpec.fromSourceFilename('buildspec.yaml'),
        environment: {
          buildImage: codebuild.LinuxBuildImage.STANDARD_7_0,
        },
      }),
      input: sourceOutput,
      role: new iam.Role(this, 'CodeBuildRole', {
        assumedBy: new iam.ServicePrincipal('codebuild.amazonaws.com'),
        managedPolicies: [
          iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess'),
        ],
      }),

    });

    pipeline.addStage({
      stageName: 'Source',
      actions: [sourceAction],
    });

    pipeline.addStage({
      stageName: 'Build',
      actions: [buildAction],
    });

  }
}

Possible Solution

For some reason it appears the CodePipeline CDK stack is creating 3 roles for this stack, when I expected 2. A couple years back I see the addition of an actionRole parameter but it since seems to have been removed. I'm not sure what the appropriate solution here is.

Additional Information/Context

No response

CDK CLI Version

2.117.0 (build 59d9b23)

Framework Version

No response

Node.js Version

Node.js v20.9.0

OS

macOS 14.1.2 (23B92)

Language

TypeScript

Language Version

typescript@5.2.2

Other information

No response

[jesseadams]

Update: Manually attaching the AdminAccess policy to the automatically created role that I did not create seems to resolve the issue, but I have no idea how to override this magically created role.

[pahud]

Can you share your buildspec.yaml? I am guessing you probably try to cdk deploy in your buildspec.yaml but your codebuild role that executes cdk deploy does not have permission to ssm:GetParameter on parameter/cdk-bootstrap/hnb659fds/version.

Generally, this is required for the role that executes cdk deploy. If you create your own codepipeline and codebuild like that, you would need to add required permissions to the codebuild role.

[jesseadams]

@pahud

You are correct.

version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: latest
  install:
    commands:
      - npm install aws-cdk-lib typescript
  build:
    commands:
      - npx cdk deploy example-stack

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[jesseadams]

Please keep this open

[amansharma-eb]

Hello @jesseadams, did you able to resolve this issue? I am also trying something similar by creating a role in cdk construct and passing the role in codebuild project and then in cdk deploy running in buildspec. I am facing the same authentication error like you.

[jesseadams]

@amansharma-eb I am sorry but no we did not. We ended up switching to terraform for this specific project.

[amansharma-eb]

Got it, Thanks!

[pahud]

If you run npx cdk deploy in the codebuild, what's happening behind the scene is that the codebuild execution role will assume multiple roles for different operations such as assets uploading, context lookup and cfn execution and you will need to allow this codebuild role to assume to those roles which were created in cdk bootstrap. We discussed this use case in CDK Live YouTube video and I had a sample code in the screenshot like this at 44:55". Please let me know if it works for you.

[amansharma-eb]

Thanks @pahud for the suggestion. I did the same thing by adding below section in my code

codePipelineTrigger.role?.addToPrincipalPolicy(new iam.PolicyStatement({
            actions: ['sts:AssumeRole'],
            resources: [
                'arn:aws:iam::123456789:role/cdk-hnb659fds-deploy-role-123456789-us-east-1',
                'arn:aws:iam::123456789:role/cdk-hnb659fds-cfn-exec-role-123456789-us-east-1',
                'arn:aws:iam::123456789:role/cdk-hnb659fds-lookup-role-123456789-us-east-1',
                'arn:aws:iam::123456789:role/cdk-hnb659fds-file-publishing-role-123456789-us-east-1',
                'arn:aws:iam::123456789:role/cdk-hnb659fds-image-publishing-role-123456789-us-east-1'
            ]
        })
    )

still while running npx cdk deploy it is referring to default codebuild roles. Am I missing something? The flow is that via the github workflow I am running aws codebuild start-build ... to which I am passing codebuild cdk-construct from s3 bucket the role I am passing in the workflow have have only codebuild and s3 bucket permission. Then in the codebuild stack from the buildspec I am running the cdk deploy to deploy the codepipeline in different stack.

[adamsar]

Is there any progress on this? I'm running into the same issue.

I'm genuinely confused though, because I have a JVM based stack that does pretty much the same thing, but my node one is only one that fails.

[adamsar]

As an update for people who come across this on Google. In the end I had to let cdk create the role, and then use escape hatches to add and edit the policies for the created one.