Auto generate types

[ejholmes]

CloudFormation now has a published JSON schema of all the cloudformation types and their inter-dependencies: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html

Would be great if troposphere could just autogenerate types based on the document.

[markpeek]

Ahhh, they published it. We gave them some input into the spec so I'll have to see what was finally decided. But yes, the plan is to have parts of troposphere be auto-generated from this doc (similar to how awacs works).

[gehadshaat]

It would be nice to generate code in a way to allow autocomplete to work with pycharm so we can see what arguments a Resource requires.

[jdreaver]

FYI I completed porting my Haskell CloudFormation library to use the JSON spec: https://github.com/frontrowed/stratosphere. There are still a handful of bugs in the official spec:

• Missing properties
• Things marked as not required when they are required
• Incorrect types
• One missing resource property

I have an open ticket in AWS business support to resolve these issues, and they'll upload a new spec soon. Overall though, it's an awesome document. No more having to manually add CloudFormation features one-by-one!

[markpeek]

@jdreaver Thanks for trailblazing on the auto-gen front and for giving me a heads up on the spec bugs. One thing that's biting me is namespace when there is a Property and Resource of the same name (EC2 NetworkInterface and SNS Subscription). I just mentioned this to the product managers here at re:invent so it is now on their radar. Not sure if this hit you or not. But agreed, having the spec for auto-gen is awesome!

[jdreaver]

No problem!

Regarding naming: I decided to use the full resource name minus AWS:: and colons/dots for my names. For example, AWS::ApiGateway::ApiKey.StageKey becomes ApiGatewayApiKeyStageKey. This is definitely a little verbose, but it does the job. Using this scheme, the only naming conflict I got was between the AWS::RDS::DBSecurityGroupIngress resource and the AWS::RDS::DBSecurityGroup.Ingress property. I just called the latter RDSDBSecurityGroupIngressProperty. I'd imagine in Python you can do something more hierarchical too, like RDS.DBSecurityGroup.Ingress.

[mwildehahn]

+1 this would be awesome

[mwildehahn]

i'd love to have access to the Documentation link for each property within stacker

[markfink]

we upvoted this feature back in November and now we do not want to wait any longer. Anybody working on this already? I researched the topic and implemented a suitable parser so we can have region specific templates and the resource elements are generated from AWS cfn resource specifications.

I propose to configure the region like this:

t = Template(region='eu-west-1')

I intend to also support properties update-type Mutable / Immutable so this can be validated prior to a deployment.

I think 100% backwards compatibility will be an issue regarding generated modules and imports from templates. So I expect that people who are going to use this feature in their templates are going to update their templates to a) set the region b) remove some imports. Question is are you going to entertain a pull request for this?

[markpeek]

@markfink I have written some code for this feature but my real job has been keeping me busy. My intent was to try to auto-generate the same style files from the resource specifications (like the way awacs does it). The thing I've been mulling in my head is the best way to handle class/resource specific validators (edit in-line, pull them from a separate file structure, or have a different validator registry). The reason for keeping them static is to ensure a common release, versioning, and user experience. Your comment seems to imply a more dynamic nature, or am I not reading your comments correctly? I'd be open to a PR but I'd like to have the discussion about direction to ensure we're on the same page for what we want from it. To your other point, it might make sense to first make a change to ensure compatibility with the current functionality and then layer on the update-type afterwards.