Closed growak closed 1 year ago
I believe when you cdk bootstrap
the account 222222222222
, you need to pass --trust
and --trust-for-lookup
for 111111111111
as described in the doc. Does this work with you?
Everything works fine when the resource is in a normal Stack and fails when it is in a Nested Stack. All accounts have been bootstrapped correctly.
@growak Thank you for your immediate response. Are you able to provide a small working sample that I can reproduce in my account? This will help us address this issue much easier.
Do you want a code repository or can I copy paste the code in comments?
@growak Please feel free to just copy paste the minimal required code in the comments or in the issue description with code blocks, I will copy/paste into my IDE and try reproduce this error.
flow.ts:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Repository, IRepository } from 'aws-cdk-lib/aws-codecommit';
import { ComputeType } from 'aws-cdk-lib/aws-codebuild';
import { CodePipeline, CodePipelineSource, ShellStep, ManualApprovalStep, IFileSetProducer } from 'aws-cdk-lib/pipelines';
export interface FlowStackProps<S extends cdk.Stage, SP extends cdk.StageProps> extends cdk.StackProps {
rolePolicy: iam.PolicyStatement[]
selfMutation: boolean
repositoryName: string
repositoryPath?: string
subRepositoryNames?: string[]
repositoryBranch: string
repositoryClone: boolean
installCommands?: string[]
commands?: string[]
stagesProps: SP[],
stageFactory: (s: Construct, n: string, p: SP) => S;
}
export class FlowStack<S extends cdk.Stage, SP extends cdk.StageProps> extends cdk.Stack {
constructor(scope: Construct, id: string, props: FlowStackProps<S, SP>) {
super(scope, id, props)
// create repositories
const repository = Repository.fromRepositoryName(this, 'Repository', props.repositoryName)
const repositoryPath = props.repositoryPath ? props.repositoryPath : '.'
const temporaryPath = '../tmp'
// create sub repositories
const subRepositories: IRepository[] = []
const subInputSources: Record<string, IFileSetProducer> = {}
if (props.subRepositoryNames) {
for(let subRespositoryName of props.subRepositoryNames) {
const subRepository = Repository.fromRepositoryName(this, `${this.capitalize(subRespositoryName)}SubRepository` , subRespositoryName)
subRepositories.push(subRepository)
subInputSources[`${temporaryPath}/${subRespositoryName}`] = CodePipelineSource.codeCommit(subRepository, props.repositoryBranch)
}
}
// create role policy
const rolePolicy = props.rolePolicy.concat(props.stagesProps.map(props => new iam.PolicyStatement({
actions: ['sts:AssumeRole'],
// change that to a more precise role generated by CDK
resources: [`arn:aws:iam::${props.env ? props.env.account: '*'}:role/*`]
})
))
// create the pipeline
const installCommands = props.installCommands? props.installCommands : []
const commands = props.commands? props.commands : []
const primaryOutputDirectory = `${repositoryPath}/cdk.out`
const pipeline = new CodePipeline(this, 'Pipeline', {
selfMutation: props.selfMutation,
dockerEnabledForSelfMutation: true,
dockerEnabledForSynth: true,
publishAssetsInParallel: true,
crossAccountKeys: true,
codeBuildDefaults: {
buildEnvironment: {
privileged: true,
computeType: ComputeType.LARGE,
},
rolePolicy: subRepositories.map(
r => new iam.PolicyStatement({
actions: [ "codecommit:GitPull" ],
resources: [ r.repositoryArn ]
})
).concat(rolePolicy)
},
selfMutationCodeBuildDefaults: {
buildEnvironment: {
computeType: ComputeType.LARGE,
}
},
assetPublishingCodeBuildDefaults: {
buildEnvironment: {
computeType: ComputeType.LARGE,
}
},
synth: new ShellStep('Synth', {
input: CodePipelineSource.codeCommit(repository, props.repositoryBranch, {
codeBuildCloneOutput: props.repositoryClone,
}),
additionalInputs: subInputSources,
installCommands: installCommands.concat([
`rm -rf ${temporaryPath}`
]),
commands: commands.concat([
`cd ${repositoryPath}`,
'npm ci',
'npm run build',
'npx cdk synth'
]),
primaryOutputDirectory
})
});
// add stages
props.stagesProps.forEach((stageProps, index) => {
const stageName = stageProps.stageName ? stageProps.stageName : `Stage${index}`
const stage = pipeline.addStage(props.stageFactory(this, stageName, stageProps));
if (index < props.stagesProps.length - 1) {
stage.addPost(new ManualApprovalStep(`Manual Approval`))
}
})
}
capitalize(str: string) {
return str.charAt(0).toUpperCase() + str.slice(1);
}
}
main.ts:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { RootCoreStack, GlobalCoreStack, RegionalCoreStack } from '../core'
import * as r53 from 'aws-cdk-lib/aws-route53';
export interface MainStageProps extends cdk.StageProps {
rootAccountId: string
domainName: string
otherDomainNames?: [string]
stageName: string
vpcCidr: string
}
export class MainStage extends cdk.Stage {
constructor(scope: Construct, id: string, props: MainStageProps) {
super(scope, id, props);
const rootCore = new RootCoreStack(this, 'RootCore', {
domainName: props.domainName,
stageName: props.stageName,
vpcCidr: props.vpcCidr,
env: {
account: props.rootAccountId,
region: 'us-east-1'
},
})
const globalCore = new GlobalCoreStack(this, 'GlobalCore', {
domainName: props.domainName,
stageName: props.stageName,
vpcCidr: props.vpcCidr,
env: {
region: 'us-east-1'
},
crossRegionReferences: true,
})
const regionalCore = new RegionalCoreStack(this, 'RegionalCore', {
domainName: props.domainName,
stageName: props.stageName,
vpcCidr: props.vpcCidr,
crossRegionReferences: true,
})
}
}
core.ts:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as r53 from 'aws-cdk-lib/aws-route53';
import { GlobalNetworkStack, RegionalNetworkStack } from './network'
export interface CoreStackProps extends cdk.StackProps {
domainName: string
stageName: string
vpcCidr: string
}
export class RootCoreStack extends cdk.Stack {
public rootHostedZone: r53.IHostedZone
constructor(scope: Construct, id: string, props: CoreStackProps) {
super(scope, id, props)
const rootDomainName = props.domainName
this.rootHostedZone = r53.HostedZone.fromLookup(this, 'RootHostedZone', {
domainName:rootDomainName
})
}
}
export class GlobalCoreStack extends cdk.Stack {
public network: GlobalNetworkStack
constructor(scope: Construct, id: string, props: CoreStackProps) {
super(scope, id, props)
this.logging = new LoggingStack(this, 'Logging', {})
this.network = new GlobalNetworkStack(this, 'Network', {
domainName: props.domainName,
stageName: props.stageName,
})
}
}
export class RegionalCoreStack extends cdk.Stack {
public network: RegionalNetworkStack
constructor(scope: Construct, id: string, props: CoreStackProps) {
super(scope, id, props)
this.network = new RegionalNetworkStack(this, 'Network', {
domainName: props.domainName,
stageName: props.stageName,
vpcCidr: props.vpcCidr
})
}
}
network.ts:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as r53 from 'aws-cdk-lib/aws-route53';
import * as cm from 'aws-cdk-lib/aws-certificatemanager';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
export interface GlobalNetworkStackProps extends cdk.NestedStackProps {
domainName: string
stageName: string
}
export class GlobalNetworkStack extends cdk.NestedStack {
public stageHostedZone: r53.IHostedZone
public certificate: cm.Certificate
constructor(scope: Construct, id: string, props: GlobalNetworkStackProps) {
super(scope, id, props);
// create certificate
/*this.certificate = new cm.Certificate(this, 'Certificate', {
domainName: rootDomainName,
subjectAlternativeNames: [
`*.${rootDomainName}`
],
validation: cm.CertificateValidation.fromDns(this.rootHostedZone),
})*/
// create stage hosted zone
const stageDomainName = `${props.stageName.substring(0, 3)}.${props.domainName}`
this.stageHostedZone = new r53.HostedZone(this, 'StageHostedZone', {
zoneName: stageDomainName
})
/*new r53.NsRecord(this, `StageNSRecord`, {
recordName: stageDomainName,
zone: this.rootHostedZone,
values: this.stageHostedZone.hostedZoneNameServers || [],
ttl: cdk.Duration.minutes(5)
})*/
}
}
export interface RegionalNetworkStackProps extends cdk.NestedStackProps {
domainName: string
stageName: string
vpcCidr: string
}
export class RegionalNetworkStack extends cdk.NestedStack {
public certificate: cm.Certificate
constructor(scope: Construct, id: string, props: RegionalNetworkStackProps) {
super(scope, id, props);
// retrieve root hosted zone
// const domainName = props.domainName
// const rootHostedZone = r53.HostedZone.fromLookup(this, 'RootHostedZone', {domainName})
// create certificate
/*this.certificate = new cm.Certificate(this, 'Certificate', {
domainName,
subjectAlternativeNames: [
`*.${domainName}`
],
validation: cm.CertificateValidation.fromDns(rootHostedZone)
})*/
// vpc
const vpc = new ec2.Vpc(this, 'Vpc', {
ipAddresses: ec2.IpAddresses.cidr(props.vpcCidr),
maxAzs: 2,
subnetConfiguration: [{
name: 'Public',
subnetType: ec2.SubnetType.PUBLIC,
}],
})
}
}
I was able to reproduce this.
Need to perform AWS calls for account xxx but the current credentials are for yyy
This occurs when you need to perform a lookup within a cross-account nested stack. I'm not aware of a way you can directly fix this within your app, but I think a viable workaround could be refactoring your code such that lookups are performed only in top-level stacks, with the values passed to the nested stacks once the lookup has been properly performed
I'm also not super sure what is causing this behavior with nested stacks to begin with, I'm not terribly familiar with them
Hey Peter, thank you for your answer. I had the same issue creating VPC in a NestedStack. Were you able to reproduce that bug too? I didn't test other constructs.
Yes, creating a VPC can cause a lookup to occur (lookup stack AZs) depending on how you have it configured. This is likely what's occurring for you in this case
Thank you for your answer. Isn't that a bug? Any idea how to fix it? I may try to do a PR!
Sorry I didn't reply to this @growak, yes, this is a bug! And, I'm not really sure at all why this is occurring. If you have any ideas, a PR would be very welcome! Otherwise, we'll try to investigate and fix this when we can
I think the issue here is that the NestedStackSynthesizer does not pass the lookupRoleArn
to the synthesizeTemplate
method.
The solution is probably to somehow inherit the settings from the parent stack synthesizer.
Hi, encountered exactly the same problem on a Python and a Typescript CDK pipeline. As @corymhall mentioned, is because the NestedStackSynthesizer doesn't pass the lookupRoleArn to the synthesizeTemplate method so the inherited Stack's methods of the NestedStackSynthesizer use the local credentials instead of assuming the cdk's cross account role.
When passing the role to the synthesizeTemplate method, the NestedStack indeed assume the cross-account role and is able to perform the lookup on the target account.
Attached a PR that solves the issue.
@IgnacioAcunaF were you still interested in continuing the PR?
Hi @peterwoodworth yes. I haven't been able to upload some changes the last weeks, but I think I'll be able to do it the next one.
Thanks for the reminder
+1 to the affected users count. Watching this thread.
Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
Describe the bug
When a resource is created in a NestedStack in a cross-account pipeline, the Build/Synth phase failed with he following error message: Need to perform AWS calls for account 222222222222, but the current credentials are for 111111111111.
111111111111 is the pipeline account 222222222222 is the target account where the resource should be deployed.
If the same resource is moved to the parent stack everything works.
Expected Behavior
Resource creation should work in NestedStack in cross-account pipelines.
Current Behavior
Resource creation failed in NestedStack in cross-account pipelines.
Reproduction Steps
Create a pipeline. Create a stage in a different account. Create a NestedStack inside de stage stack. Create a resource in the Nested Stack.
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.74.0
Framework Version
No response
Node.js Version
v16.20.0
OS
Amazon-Linux 2023
Language
Typescript
Language Version
No response
Other information
No response