mingrammer / diagrams

:art: Diagram as Code for prototyping cloud system architectures
https://diagrams.mingrammer.com
MIT License
36.8k stars 2.39k forks source link

Switch the code engine to any Pulumi supported language? #1

Open ringods opened 4 years ago

ringods commented 4 years ago

Your project is awesome and really helpful. The only drawback is that your syntax stands on its own as a standalone library.

Not sure if you know Pulumi. This is a tool which allows to implement and provision cloud resources. Providers can be built for anything which has an API.

I'm wondering how easy it would be hook your graph generation to the Pulumi engine to generate diagrams of what was written in code.

mingrammer commented 4 years ago

I've heard about the Pulumi before, but I don't know the details for that tool. I need to check it.

Is this right the Pulumi engine you said?

import pulumi
from pulumi_aws import s3

# Create an AWS resource (S3 Bucket)
bucket = s3.Bucket('my-bucket')

# Export the name of the bucket
pulumi.export('bucket_name',  bucket.id)

Thank you!

ringods commented 4 years ago

I think so. Here is the list of Python packages representing the core Pulumi engine as well as the different providers:

https://pypi.org/search/?q=pulumi

mingrammer commented 4 years ago

Hm, the syntax is very different between diagrams and pulumi. So I don't know how worth that convert to pulumi-compatible syntax. (may need large effort I think, because of Clusters feature)

I need to think a lot about it.

bkmeneguello commented 4 years ago

I think Pulumi/Terraform are a different concept when compared to Diagrams. They are infrastructure recipes while the former is a more abstract diagram tool. If someone should incorporate something are they (I'm not a maintainer of this project son it's just MHO). Maybe, to ease things, an adapter could be developed to enrich the Pulumi/Terraform with some metadata to provide diagramming information (nesting and clustering, mainly and weak connections to not make the diagram a mess)

c2h5oh commented 4 years ago

Alternative idea: pulumi can export stack to json pulumi stack export > file.json - that json has all information needed to generate a diagram if someone wrote a converter.

Bonus: it would work regardless of which pulumi-supported language you used and would not require changes to this project.

mingrammer commented 4 years ago

pulumi stack export looks like a terraform graph. Right?

c2h5oh commented 4 years ago

More like values part of terraform show -json https://www.terraform.io/docs/internals/json-format.html#values-representation

mingrammer commented 4 years ago

Thank you. Let me review that it's possible.

mingrammer commented 4 years ago

Related issue: #6 terraform support

c2h5oh commented 4 years ago

@mingrammer one more note: Pulumi uses terraform plugins internally, so support for terraform state graph should handle both. I'll investigate if/how that state graph can be accessed in Pulumi and get back to you.

ringods commented 4 years ago

@c2h5oh Pulumi wraps terraform providers to be exposed as Pulumi providers. But the communication between the Pulumi engine and Pulumi providers is different from Terraform. As a result, your statement support for terraform state graph should handle both is invalid.

ringods commented 4 years ago

This is actually the same request as #6 and should be investigated together.

Gerrit-K commented 4 years ago

Pulumi also does support outputting the dependency graph as a dot file: pulumi stack graph <output-filename>. The result however includes the whole graph, so probably also a lot of things that you normally wouldn't want to put in an architecture diagram. Might be useful on this topic though.