Feature request: Allow Data Insertion/Transformation into troposphere

[LarsFronius]

In my case I transform a docker-compose file using https://github.com/ambitioninc/container-transform/ into an ECS definition and from there into the correct cloudformation structure.

I do now have the dict/list at hand needed to create my troposphere ecs.TaskDefinition(). Unfortunately troposphere only allows me to explicitly add troposphere.ecs.ContainerDefinition and troposphere.ecs.Volume under the given attributes of the ecs.TaskDefinition and otherwise throwing an "TypeError: ContainerDefinitions is <type 'dict'>, expected [<class 'troposphere.ecs.ContainerDefinition'>]"

Since troposphere does not provide an easy way for me to take my dictionary

{                       
                        "Cpu": 1,
                        "Environment": [
                            {   
                                "Name": "REGISTRY_STORAGE",
                                "Value": "s3"
                            },  
                            {   
                                "Name": "REGISTRY_STORAGE_S3_REGION",
                                "Value": "eu-west-1"
                            }
                        ],
                        "Essential": True,
                        "Image": "registry:2",
                        "Memory": 500,
                        "Name": "registry",
                        "PortMappings": [
                            {   
                                "ContainerPort": 5000,
                                "HostPort": 5000
                            },  
                            {   
                                "ContainerPort": 5001,
                                "HostPort": 5001
                            }
                        ]
                    }

and turn it into a valid troposphere.ecs.ContainerDefinition, I had to open up the validation as you can see in this commit https://github.com/Inbot/troposphere/commit/c879500055535323c5eadb5689cd1bb870c051ee It's not a very secure implementation and allows for any data structure to be added to the cloudformation template, which is probably not what we want, but works for me.

I'd love to see a way to transform data/json into valid troposphere objects. Any thoughts on this?

[markpeek]

Interesting question on how to bypass some of the troposphere type checking. I think this might be a better way although I would like to get input from you (plus from @phobologic and @bobveznat):

diff --git a/troposphere/__init__.py b/troposphere/__init__.py
index 31957db..ac09493 100644
--- a/troposphere/__init__.py
+++ b/troposphere/__init__.py
@@ -148,6 +148,12 @@ class BaseAWSObject(object):
     def validate(self):
         pass

+    @classmethod
+    def from_dict(cls, title, d):
+        obj = cls(title)
+        obj.properties.update(d)
+        return obj
+
     def JSONrepr(self):
         for k, (_, required) in self.props.items():
             if required and k not in self.properties:

Example:

import troposphere.ecs as ecs
from troposphere import Template

d = {
    "Cpu": 1,
    "Environment": [
        {
            "Name": "REGISTRY_STORAGE",
            "Value": "s3"
        },
        {
            "Name": "REGISTRY_STORAGE_S3_REGION",
            "Value": "eu-west-1"
        }
    ],
    "Essential": True,
    "Image": "registry:2",
    "Memory": 500,
    "Name": "registry",
    "PortMappings": [
        {
            "ContainerPort": 5000,
            "HostPort": 5000
        },
        {
            "ContainerPort": 5001,
            "HostPort": 5001
        }
    ]
}

t = Template()
cd = ecs.ContainerDefinition.from_dict("mycontainer", d)
td = ecs.TaskDefinition(
    "taskdef",
    ContainerDefinitions=[cd],
    Volumes=[ecs.Volume(Name="myvol")],
)
t.add_resource(td)
print t.to_json()

It seemed to work fine (json inspection) when I did a quick smoke test on both an AWSObject and AWSProperty. Just need to decide if we really want this as part of the API.

[LarsFronius]

Hey @markpeek

for me, this approach is perfectly fine. This way I am able to generate cloudformation templates/part of templates programmatically from other data sources and integrate them with the troposphere dsl, including all of the validation of troposphere still being in place.

:+1: real good!

[phobologic]

Seems fine to me.

On Thu, Aug 13, 2015 at 12:44 PM Lars Fronius notifications@github.com wrote:

Hey @markpeek https://github.com/markpeek

for me, this approach is perfectly fine. This way I am able to generate cloudformation templates/part of templates programmatically from other data sources and integrate them with the troposphere dsl, including all of the validation of troposphere still being in place.

[image: :+1:] real good!

— Reply to this email directly or view it on GitHub https://github.com/cloudtools/troposphere/issues/293#issuecomment-130817817 .

[markpeek]

Feature for this has been merged in. Thanks.