search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.66k stars 3.92k forks source link

Nested stack support #239

Closed eladb closed 5 years ago

eladb commented 6 years ago

It should be easy to define nested stacks in the CDK.

This requires a good story behind uploading synthesized templates to S3 and "plugging in" the URL of the nested stack into the parent stack (see #233).

We should have a story on how to reference resources from the parent child in the child stack (i.e. using nested stack "Parameters") and how to reference resources from the child stack in the parent stack (i.e. using "Outputs").

tvb commented 6 years ago

@eladb status? I need nested stack support before I can start using this cdk really.

eladb commented 5 years ago

This should not be too hard to support through assets.

eladb commented 5 years ago

A bit more context/direction/pointers/ideas:

As a user, I'd like to be able to be able to define a nested stack like any other stack in the CDK:

class SmallStack extends Stack {
  // ...
}

class BigStack extends Stack {
  constructor(p: Construct, id: string) {
    new SmallStack(this, 'NestedStack');
  }
}

const app = new App();
new BigStack(app, 'my-awesome-stack');
app.run();

An instance of SmallStack has been added as a child for BigStack. My expected behavior for this would be that SmallStack will be modeled as a nested in the my-awesome-app template.

This technically means that:

  1. The CDK app should synthesize two CloudFormation templates: one for my-awesome-app (like today) and the other for the nested stack.
  2. The template my-awesome-app will include an AWS::CloudFormation::Stack resource with TemplateURL assigned from a CloudFormation Parameter.
  3. Before deploying my-awesome-app, the toolkit should upload the nested stack template to S3 and then assign the location of the uploaded template to the parameter wired to the TemplateURL.

Assets: the protocol between the CDK and the toolkit (defined under the cx-api module) allows CDK apps to emit metadata entries that instruct the toolkit to package and deploy various types of assets (s3 files, s3 directories, docker images) and then reference them through CloudFormation parameters when the stack is deployed, which is effectively what we need here.

sam-goodwin commented 5 years ago

This has implications on how Stacks are instantiated, render toCloudformation and import Output values.

Minor:

How to distinguish the behavior of aStack instance when used as a root vs nested stack?

If it's a nested stack, we want to render both its template and the AWS::CloudFormation::Stack resource, passing in parameter values from the parent. Can/should a stack be designed for both use-cases? 

interface ChildStackProps extends cdk.StackProps {
  bucketArn?: string; // what does it mean to specify this as a root stack?
}

class ChildStack extends Stack {
  constructor(parent: cdk.Construct, name: string, props: ChildStackProps = {}) {
    super(parent, name, props);

    this.bucketParameter = new cdk.Parameter(this, 'BucketArn', {
      type: 'AWS::S3::Bucket'
    });
    if (props.bucketArn) {
      // how to 'accept' the bucketArn when used as a nested stack?
    }
  }

  // how does toCloudformation distinguish its behavior?
}

Parent stacks import a nested stack's Output value with Fn::GetAtt Outputs.NestedStackOutputName, not Fn::ImportValue.

Do we expect developers to use a different import API when using nested stacks, or re-purpose the existing import and XXXRef convention? Passing around BucketRef works well when the stacks are all root stacks, but the behavior is different for nested stacks, so developers must be aware of where the value came from.

class ParentStack extends Stack {
  constructor(parent: cdk.Construct, name: string) {
    super(parent, name);
    this.child = new ChildStack(this, 'Child');
    // Can I do this? It will use Fn::ImportValue instead of Fn::GetAtt
    const bucket = s3.BucketRef.import(this, 'Bucket', this.child.bucketProps);
  }
}

class ChildStack extends Stack {
  public readonly bucketProps: s3.BucketRefProps;
  constructor(parent: cdk.Construct, name: string) {
    super(parent, name);
    this.bucketProps = new s3.Bucket(this, 'Bucket').export();
  }
}

How does import work for two child stacks in a single hierarchy?

I don't know if two child stacks can import values from each-other, or if they must be proxied through Fn::GetAtt and Parameters via a common parent stack:

class ChildStack extends Stack {
  public readonly bucketProps: s3.BucketRefProps;
  constructor(parent: cdk.Construct, name: string) {
    super(parent, name);
    this.bucketProps = new s3.Bucket(this, 'Bucket').export();
  }
}

class ChildStack2 extends Stack {
  constructor(parent: cdk.Construct, name: string, child: ChildStack) {
    super(parent, name);
    // will this work?
    s3.BucketRef.import(this, 'Bucket', child.bucketProps);
  }
}
eladb commented 5 years ago

Good points. All the cross reference aspects should probably be designed in conjunction with @rix0rrr's work on #1324. Rico proposes to eliminate the need for explicit import/export, which might serve to identify which type of reference this is: between two root stacks, between two nested stacks or between a nested stack and it's parent.

The bucketArn input prop example you provided above may be make complete sense as a root stack as well as a nested stack. A stack subclass is a reusable component that can be instantiated many times within the same app or in different apps, and it may accept inputs. Bear in mind that there should probably be a distinction between a concrete value for bucketArn and a value that includes tokens (cdk.unresolved(props.bucketArn) returns true). In the former case, the value would just propagate naturally to the nested/root stack without any need to utilize deploy-time resolution via CloudFormation parameters. In the latter, tokens must first be resolved (at deploy-time) and only then they can be used, so "magic" need to happen.

All the issues you bring up are relevant and important. However, try to design this in layers. How can we provide support for nested stack with minimal magic and maximum control for the user (can we provide only runtime errors if users attempt to cross the boundaries in the wrong way?). Then, we can see how we can sprinkle magic on top...

Meta: would probably be easier to discuss this design via PR under design/*.md (see Design Process).

tyron commented 5 years ago

Keep in mind that users would expect cdk diff to traverse into the nested stacks and provide a complete list of differences of the application. This is even not supported on CloudFormation yet.

dougmoscrop commented 5 years ago

@tyron yeah it sure isn't, and not only that it basically always returns changes even when nothing has changed.

as a workaround I created some utilities here https://github.com/node-cfn

it implements a poor-mans diff, supporting nested stacks - at least as best as I can. it's probably not as precise as a solution would need to be to meet aws standards; it's a library and I use it in a serverless plugin (https://github.com/dougmoscrop/serverless-plugin-bootstrap) which is responsible for representing the diff textually

I would love to switch that plugin to use the cdk under the hood instead if it could support nested stacks. I don't know if any of my work there could be useful.

Approach wise, it topologically sorts the root stack and then runs a diff on each nested stack, and marks changes if it detects a change in things like the template URL (which is always hashed), or any properties in actual templates themselves, and then any thing that depends on a thing that changed gets marked as changed too (so it can possibly err on the side of falsely marking something as changed, but then the visual diff will reveal that nothing really changed). But it avoids false positives when there actually are no changes.

it doesn't currently support imports or nested stacks within nested stacks, though those could be added.

bverhoeve commented 5 years ago

@eladb

Hi everyone,

Is there an update on the status of this issue? I'm using CDK for a rather complex set of stacks, which will probably go over the resource limit of CloudFormation, so I'm wondering if I need to foresee issues or if there is already an out-of-the-box solution from CDK.

eladb commented 5 years ago

@bverhoeve i am working on official support for nested stacks in #2821 but I am not even sure you will need this. You can define any number of stacks in the CDK and freely reference resources between them. The cdk will automatically synthesis exports and imports (as long as they are in the same environment). This is mostly the same as nested stacks (some would say superior since there could be more complex relationships).

bverhoeve commented 5 years ago

@eladb thank you for the update, this sounds like great news!

So if I get this correctly, this would mean that I can define multiple Stack objects on which I can define up to 200 CloudFormation resources and CDK will split in multiple CloudFormation templates and take care of the imports/exports?

eladb commented 5 years ago

Yes, that's already supported (docs)

SteveHoggNZ commented 5 years ago

I think nested stacks could be useful for our use case:

I figure that if we do this with nested stacks then those stacks will manage the creation or new alarms and deletion of the old ones in an automated way.

stephenh commented 5 years ago

@eladb FWIW the resource_stack link you gave is great, but it took me quite awhile to find it (I nearly opened an issue asking if this capability [cross stack references] was even supported, b/c it really seems like it should be, but afaict none of the howtos pointed it out, and none of the readmes/etc. I'd found explicitly used that approach, and after combing through issues ended up here.)

Would be great if this multiple stack how to, which is currently just "I'll make multiple versions of the same stack" had a little link at the bottom for "btw if you want to do cross-stack references, see [the link you just gave]".

pierreis commented 5 years ago

@eladb I see that the PR #2821 you gave mentioned earlier has been closed. Does it mean that the development has been put on hold? I think it would be incredibly useful for constructs such as DynamoDB Global Tables, which generate stacks and at the moment yield parameter errors when used inside a stack.

kbessas commented 5 years ago

@eladb Is there any update on the status for this? The PR has been re-opened which sounds like good news. Is it reasonable to expect nested stacks released within October or November?

Despite the suggestion for using multiple top-level stacks, nested stacks are a completely different ball game within CloudFormation.

The main problem with multiple top-level stacks is that if a stack fails to update, you do not get the waterfall rollback of all the stacks that have been changed. In some cases, this can lead to really big issues!

eladb commented 5 years ago

Hey @kbessas, I hope I'll be able to follow up on this soon. Generally I'd like to land this for the exact reasons you described.

crucialfelix commented 5 years ago

I'm a bit confused now. Maybe my confusion will help someone...

Currently (1.9.0) if I try nested stacks (

and cdk synth that, the top level ns-dev.template.json is just {} — so I assume it is not possible right now. It cannot deploy.

I tried to use a Construct as the main (ns-dev) that contains the sub-stacks. This doesn't seem to work either:

ns-cdk-devops ❯ cdk deploy
Since this app includes more than a single stack, specify which stacks to use (wildcards are supported)
Stacks: nsdevnetE5343A5C nsdevappcacheEE591D45 nsdevdb84DB0F47 nsdevappecs4B1E5DD6 nsdevappcdn9B80D361

ns-cdk-devops ❯ cdk deploy nsdevnetE5343A5C                                                                                                                                                                                                                                  
nsdevnetE5343A5C: deploying...
nsdevnetE5343A5C: creating CloudFormation changeset...

 ❌  nsdevnetE5343A5C failed: ValidationError: Stack [nsdevnetE5343A5C] does not exist
Stack [nsdevnetE5343A5C] does not exist

I guess "does not exist" means that the template it isn't uploaded to aws yet?

So at the moment, multiple stacks can only be up at the app level, and any sub functionality inside those should be written with Construct. Is that about right?

tvb commented 5 years ago

@eladb status? I need nested stack support before I can start using this cdk really.

One year later it's here 💪

eladb commented 5 years ago

Well the issue is closed... we merged this feature yesterday and expect it to be released with the next release (up to two weeks).

ekzGuille commented 5 years ago

Hi, I'm trying to create a CloudFormation template which uses nested stacks.

First of all I have to say that I don't use cdk deploy in order to deploy my template. What I do is generate .json templates inside the code calling app.synth() and executing .js file. (Those .json are going to be "manually" stored in a S3 which belongs to a different account from the one which is deploying the main template).

I've made some tests using NestedStack construct and CfnStack class and what I found out is:

I'm mostly sure that my template will have more than 200 CF resources so I need to use nested stacks, but the problems are:

(I have to insert the S3 Bucket url because as I mentioned before they will be stored in a specific location.)

I've seen https://github.com/aws/aws-cdk/issues/239#issuecomment-518978759 but does not solve my problem.

FYI I'm using @aws-cdk (v1.15.0) and Typescript