Use Case: Use cloudformation-ruby-dsl as a library

[harlanbarnes]

I'm not sure if I'm going to explain this well, but I'll give it a shot.

I use cloudformation-ruby-dsl in a slightly different way. I "add features" to it by the fact that it is ruby script and can do extra things like "lookup the latest AMI ID" or "read a YAML file for more information". As such, I pass in command-line arguments and parse them using a rubygem (mixlib-cli in my case, but it could be anything.)

As a side note, prior to aws-sdk I wasn't able to do this because all of ARGV got passed directly to the old cfn tools (at which case, it would fail because of it not recognizing CLI options.) At the moment, I can ALMOST get it to work. The quirk is that my (the "outer") parsing of CLI options has to be able to handle any options that the cloudformation-ruby-dsl requires (i.e. --stack-name has to be handled by my options parsing ... but this is manageable) ... while using any CLI options that cloudformation-ruby-dsl doesn't support can cause problems (so far, the biggest problem I've found is that diff action seems to pass in all of ARGV that the cloudformation-ruby-dsl doesn't recognize to the Diffy object as options ... which causes Diffy to not work.)

I guess at the heart of it, I'd want to be able to use the template DSL more like a library ... but, of course, let others use it as a pure standalone script too.

Is there a way to do this that I am just missing?

[jonaf]

Hi @harlanbarnes ! Thanks for your interest. We actually refactored and released 1.0 of the Cloudformation Ruby DSL in order to support this use-case, or at least get us closer. In fact, it's quite possible you can circumvent the command line interface by directly instantiating a new TemplateDSL instance (import dsl.rb instead of cfntemplate.rb). Does that get you closer, or is the argparsing still getting in your way?

[harlanbarnes]

Hey @jonaf, thanks for the idea. I've tried this out, but honestly, this moves me past my ruby knowledge. Here's a gist example with the errors I'm getting.

Am I correct in understanding that if I can sort this part out, I'd still have to handle implementing the actions (i.e. create, update, diff) themselves? Since that part hasn't be refactored out yet?

Again, thanks for all y'all do on this.

[joshuatobin]

@jonaf Any chance you could provide some examples? I'm looking for the use case to generate the template by using it within a class. Thanks for the great work on this!

[jonaf]

@harlanbarnes and @joshuatobin : The easy way to make this work is to use a Proc or code block rather than a lambda. Lambdas check the argument list (a la methods), whereas Proc and blocks will simply apply nil's wherever an argument is expected.

Here's an example of this in action. You can do whatever you want with your template block. In fact, because it ultimately is a Proc, you can store and mutate it to your heart's content before actually sending it through to the DSL for processing.

#!/usr/bin/env ruby

require "cloudformation-ruby-dsl/dsl"

myTemplate = TemplateDSL.new(
    {},
    'jona-cfn-test',
    'us-east-1',
    false,
    &proc {
      tag :'With Spaces' => 'BeforeAfter'
      tag :NoSpaces      => 'AllTogether'

      resource 'jona-cfn-test',
        :Type       => 'AWS::EC2::Instance',
        :Properties => {
          :ImageId        => 'ami-1ccae774',
          :InstanceType   => 't1.micro',
          :SecurityGroups => ['default'],
          :KeyName        => 'test-keypair',
          :Tags           => [{ Key: 'Name', Value: 'jona-cfn-test' }]
        }
    }
  )

puts myTemplate.to_json

[flyinbutrs]

This is so close to how I want to use it, but I can't quite get what I need. What I'm looking for is to build up a library of functions that can be manipulated similar to ruby syntax, with the end result being a piece of a template. And then, I'd like to be able to add these results together to form a larger template.

The TemplateDSL class is close, but in order to "add" different parts together, I'd have to take the output json, convert it to a hash, deep merge the hashes together, and then the end result is my template. It's doable, but it's clunky.

Also, the syntax of it is really kludgy. What are the purpose of those first 4 parameters in the context creating a representative object for manipulation? Those details could be useful further down the line, but at the basic object level, it's not that useful.

EDIT: Apologies for the rant. :)

[jonaf]

@flyinbutrs I haven't tried this, but I don't see any reason why you couldn't build a proc by merging new additions to it, and then passing it as an executable block to TemplateDSL.new.

I can see an argument for hoisting TemplateDSL as an abstract class taking only a Proc as input and calling it something else, like AbstractDSL, and then adding the option-specific stuff on top, so that the abstract DSL class can be used separately from everything else, as a library or low-level entry point to generating a template but using your own set of options / management. But is it really too hard to use it in its current state? Maybe you can provide an example of how you'd like it to work?

[flyinbutrs]

Hey, sorry, I was just about to come and rewrite that. I think I'm more frustrated with myself than anything, because I can't quite get it to work the way I want, and I don't think I understand the ruby well enough to make it do what I want.

I guess what I'm trying to do is build a standard design of various CF objects, and let other coworkers relatively easily combine the predefined parts into the desired infrastructure. As such, most parameters need to be overridable, but if you're doing standard everything it should be an extremely concise script.

[SaltwaterC]

I'm also interested in using cloudformation-ruby-dsl as both library and it's fairly standard CLI behaviour. While I've monkey-patched the code to be able to name the templates and hook an event logger (exit(true) - I'm talking about you), my grief is with the lack of actions in the TemplateDSL object since everything action wise is wired into the cfn method.

By moving stuff from cfn to TemplateDSL nothing is lost on the CLI side as cfn receives a TemplateDSL object as argument, but it would make life a lot easier on the library side of the things. I'll make cfn more readable as well as the case statement would be trivial.

I could make these changes, but I'd discuss them first instead of sending a PR and see it rot in the PR tab.

[SaltwaterC]

Actually, the whole monkey-patch to cover my basic use case as library is as simple as:

class TemplateDSL
  def name(stack_name)
    @stack_name = stack_name
  end

  def cfn_client
    if @cfn_client.nil?
      @cfn_client = AwsCfn.new(
        region: @aws_region,
        aws_profile: @aws_profile
      ).cfn_client
    end

    @cfn_client
  end

  def stack_options
    {
      stack_name: @stack_name,
      template_body: JSON.generate(self),
      parameters: {},
      tags: {},
      capabilities: %w(CAPABILITY_IAM)
    }
  end

  def action_create
    cfn_client.create_stack stack_options
  end

  def action_update
    cfn_client.update_stack stack_options
  end
end

YMMV as I don't use parameters or tags.

[temujin9]

@SaltwaterC This feels like a simple and useful improvement. Could I encourage you to turn it into a pull request (modifying the original code, rather than monkey-patching)?