Example - Depend on existing stack resource

[dextermarkley]

First off, I want to say thank you for creating this cf-ruby. It has made my life much easier.

I recently used terraform on a project and terraform has a concept of depending on existing stacks. I thought it would be awesome to have a way for cf-ruby-dsl to support the same concept.

Usage could be something like launching all your infrastructure (vpc, subnets, routes, etc), then launching your applications into that infrastructure with this functionality.

Usage would be something like this: load_remote_resources 'base_network', 'my-existing-cloudformation-stack'

resource 'PublicSubnet', Type: 'AWS::EC2::Subnet', Properties: { VpcId: get_remote_resource_id('base_network', 'VPC'), CidrBlock: '10.0.0.0/24', }

This is my best stab at adding that functionality. I am not happy about having to instantiate an additional cfn_client, and was not sure about function names.

Is this a feature you would be interested in adding? Can you think of a better way to accomplish this task?

Thanks!

[jonaf]

This is interesting. :)

One way we handle this currently is using CustomResources backed by AWS Lambdas. In my experience, this works quite well.

Another way to handle it is to use "Outputs" from one template in another template. You can do this by creating a stack that itself creates stacks. There is one side-effect of this, though, which is that updates to one stack may propagate an update to the other, and sometimes this can have undesirable effects. It's also hard to always know exactly what impact would be experienced by the update(s).

The third and most feasible way of doing this, that I can think of, is to simply make whatever API calls you want first, and then pass those to your template. You can even write the code in your pull request inside of the template, if you like. I would love to get some community feedback on the utility of having this inside of the Ruby DSL.

As for the actual implementation of this particular example, I wonder if it would be a better idea to add a "load_external_template" method, which can be invoked outside of the template (above template do, for example), similar to the Table.load methods we already have. Then you could reference paths within it, and even add some helper functions, e.g. for mappings. It would be nice to also refactor a little bit and see if we could re-use the same AwsCfn client. I think that's doable, with a little bit of refactoring. This also has the benefit of (a) natural caching and (b) preventing race conditions where you reference a property of a stack, the stack you're pulling properties from completes an update, and you reference a property of the stack later in your template and it has an incompatible value because an update to that stack occurred in the middle. Thoughts?

[zhelyan]

Here's a similar discussion in the cfn-dsl project: https://github.com/stevenjack/cfndsl/issues/167

[jonaf]

Additionally, this recent feature of CloudFormation changes the game a bit: http://blog.flux7.com/a-review-of-aws-cloudformation-cross-stack-reference

[zhelyan]

Yes, and as you said there are many ways to link / orchestrate a bunch of stacks. I'd vote for https://github.com/bazaarvoice/cloudformation-ruby-dsl/issues/57 which should make it simpler. People can then add a thin "define my infrastructure" config layer on top where they do all this dance, pass parameters etc. and hand over actual stack cteation/update etc to the lib.

[jonaf]

This has been open for several months with no activity. I'm going closing it in favor of whatever shakes out of #57 . We can re-open if there's a solid argument otherwise.