Open alexandrnikitin opened 4 years ago
Inter-cluster edge is not supported now. Cluster class is for just grouping the nodes now. Let me research if it is possible (I think it could be achieved lhead/ltail options of Graphviz)
And 3rd way is not possible because Python does not support shift operation between lists. To support this method, we should create a grouping purpose class like “NodeList” so that let someone connects the nodes between clusters.
For now, you can connect the nodes between clusters in this way: https://diagrams.mingrammer.com/docs/examples#clustered-web-services
I've played around this morning with diagrams
and have ran into a few issues with arrow direction. @mingrammer please let me know if I am doing something wrong.
From the step function, it should go into the first X. From the very last X lambda, it should go to both the MoveToOrg
lambda as well as the S3 bucket -> SNS.
I also haven't found an appealing way to represent StepFunctions
map state feature / concurrency within it
with Diagram("AcctCreation", show=False):
begin = (
SQS("KickoffQueue")
>> Lambda("AcctCreate")
>> StepFunctions("StepFunctions")
)
with Cluster("StateMchne"):
with Cluster("Cncrrntx3.1"):
lambdas = Lambda("X") >> Lambda("X")
x = lambdas >> Lambda("X") >> Lambda("X")
with Cluster("Cncrrntx3"):
z = x >> Lambda("X") >> Lambda("X")
z >> [S3("X") >> SNS("Notif"), Lambda("MoveToOrg")]
begin >> lambdas
Really enjoying this package, great job! Excited to share this with the team.
@jbpratt78 >>
operation returns last node. So the return value of Lambda("X") >> Lambda("X2")
is Lambda("X2")
, therefore begin >> lambdas
will connect the begin
to Lambda("X2")
.
In this case, nodes that directly connect to the preceding node should be declared separately. You can write like this:
with Diagram("AcctCreation", show=False):
begin = (
SQS("KickoffQueue")
>> Lambda("AcctCreate")
>> StepFunctions("StepFunctions")
)
with Cluster("StateMchne"):
with Cluster("Cncrrntx3.1"):
entry_lambda = Lambda("X")
out_lambda = Lambda("X")
entry_lambda >> out_lambda
inter_lambdas = out_lambda >> Lambda("X") >> Lambda("X")
with Cluster("Cncrrntx3"):
post_lambdas = inter_lambdas >> Lambda("X") >> Lambda("X")
store = S3("X")
store >> SNS("Notif")
post_lambdas >> [store, Lambda("MoveToOrg")]
begin >> entry_lambda
This issue now seems to contain two different issues. I am getting back to the issue of arrows to "clusters".
Yes, graphviz can have arrows between groups/clusters, but it is kind of a hack: You declare an edge between two nodes and add an annotation that you want to start/end at the cluster (https://stackoverflow.com/questions/2012036/graphviz-how-to-connect-subgraphs)
If you would want to implement that in mingrammer, e.g.:
lb = ELB("lb")
services = Cluster("Services")
with services:
svc_group = [ECS("web1"),
ECS("web2"),
ECS("web3")]
lb >> services
you need to get one node from the services cluster, so that the correct code could be generated
To get the node, you either have to implicitly retrieve any one node from the cluster – so the cluster must keep all node objects (currently the node is immediately destructured when adding it to the cluster, AFAICS).
Or the node must be a assigned to a variable (e.g. svc_group_node1
and then something like:
lb >> svc_group_node1.cluster()
@kontrafiktion my apologies, this was due to my lack of understanding as I thought my cluster usage was related. Thank you for the help @mingrammer
I'm with @kontrafiktion on this one. I would also like to see arrows between clusters. But allow also for the more Pythonic way of writing this:
lb = ELB("lb")
with Cluster("Services") as services:
ECS("web1")
ECS("web2")
ECS("web3")
lb >> services
BTW, have a look at #34.
@kontrafiktion @ringods
Can't reproduce this trick. diagrams
0.6.3. Full code of script:
from diagrams import Cluster, Diagram
from diagrams.aws.compute import ECS
from diagrams.aws.network import ELB
with Diagram("Cluster link", show=False):
lb = ELB("lb")
with Cluster("Services") as services:
ECS("web1")
ECS("web2")
ECS("web3")
lb >> services
failed with
Traceback (most recent call last):
File "/home/iaro/PycharmProjects/diagrams_test/tmp.py", line 12, in <module>
lb >> services
File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 326, in __rshift__
return self.connect(other)
File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 372, in connect
self._diagram.connect(self, node, directed)
File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 168, in connect
self.dot.edge(node.hashid, node2.hashid, **attrs)
AttributeError: 'Cluster' object has no attribute 'hashid'
@IaroslavR my code snippet was a proposal of how I would like to use it. It doesn't work like that (yet).
@ringods Got it
I +1 this! When you have 2 clusters, each one accessing 2 other clusters with many components in it, it makes the diagram less readable.
Also, it prevents from linking clusters directly.
Another use case: I wanted to have a Kafka cluster and a Zookeeper cluster with 5 nodes each. All of the Kafka nodes are connected with -
and the Zookeeper nodes as well. Now, when trying to visualize that there is a connection between the Kafka and Zookeeper cluster, one can either connect the two Clusters together (preferred) or visualize the connection from each Kafka Node to each Zookeeper node, as all Kafka brokers could could theoretically connect to each Zookeeper node. The latter generates a mess on my end so the proposed solution in this issue would be a great feature.
I'll review this proposal in a week.
👍 for the feature.
I work on a tool to automate AWS infrastructure detection and draw diagram of the detected infrastructure. Right now, there's a high number of duplicates caused by redundant connections to multiple subnets from EC2 instances, Lambda functions etc.
Having this feature in place, the diagrams would look much cleaner and easier to understand.
I use a corresponding feature on PlantUML and would add my request here for this feature too. Great tool - thanks!
My current work around : Make the cluster a list, then point the middle elements together. It is a little more defining if you want multiple elements to have arrows, or just the cluster as a single arrow.
with Cluster("cluster1"):
cluster1 = [EC2("node1"),
EC2("node2"),
EC2("node3")]
with Cluster("cluster2"):
cluster2 = [EC2("node1"),
EC2("node2"),
EC2("node3")]
cluster1[1] >> cluster2[1]
@alexandrnikitin
I want to have arrows from and to Clusters, e.g.
Below is a work-around:
from diagrams import Diagram, Cluster, Edge, Node
graph_attr = {
"layout":"neato",
}
scaling_clus_attr = {
"bgcolor":"transparent",
"pencolor":"blue",
"penwidth":"4.0"
}
with Diagram("\n\nNodes on Cluster boundary\nwith connecting arrow", show=False, graph_attr=graph_attr) as diag:
with Cluster("cluster 1"):
A_UpLf = Node("", shape="plaintext", pin="true", pos="0,4")
A_LwRt = Node("", shape="plaintext", pin="true", pos="4,0")
with Cluster("cluster 2", graph_attr=scaling_clus_attr):
B_UpLf = Node("", shape="plaintext", pin="true", pos="8,3")
B_LwRt = Node("", shape="plaintext", pin="true", pos="10,1")
A_tail = Node("", shape="plaintext", pin="true", pos="4,2")
B_head = Node("", shape="plaintext", pin="true", pos="8,2")
A_tail >> B_head
diag
Is it working with some components inside the cluster? I tried your code but only ended with an arrow, no square...
@Karreg did you change the layout engine to "neato" ?
graph_attr = {"layout":"neato", }
Here is another way. It only works with the 'dot' layout engine, which is the default for this library, but I explicitly specified it for clarity.
The main thing to notice is that this library automatically sets the name
attribute of the Cluster
to be "cluster_" + self.label
.
See https://github.com/mingrammer/diagrams/blob/dfd8e0a52c8c4d1c3ce95dc7161c23bb2eaf0acb/diagrams/__init__.py#L223
You must keep this in mind when referring to the Cluster
name
, in the ltail
and lhead
attributes of Edge
.
Below is a replication of Figure 22 from https://www.graphviz.org/pdf/dotguide.pdf using this library. This is a good example of how the Edge
's head and tail are truncated at the Cluster
boundary.
from diagrams import Diagram, Cluster, Edge, Node
graph_attr = {
"layout":"dot",
"compound":"true",
"splines":"spline",
}
node_attr = {
"shape":"ellipse",
"height":"0.8",
"labelloc":"c"
}
with Diagram("\nCluster to Cluster Edges", show=False, direction="TB", graph_attr=graph_attr) as diag:
with Cluster("0"):
a = Node("a", **node_attr)
b = Node("b", **node_attr)
c = Node("c", **node_attr)
d = Node("d", **node_attr)
with Cluster("1"):
e = Node("e", **node_attr)
f = Node("f", **node_attr)
g = Node("g", **node_attr)
h = Node("h", **node_attr)
a >> [b, c]
[b, c] >> d
e >> [f, g]
b >> Edge(lhead="cluster_1") >> f
c >> Edge(ltail="cluster_0", lhead="cluster_1") >> g
c >> Edge(ltail="cluster_0") >> e
d >> [e, h]
diag
To have a more meaningful Cluster
label, but still keep the Cluster
name simple, just add the Cluster
keyword argument graph_attr
dictionary, and then set the "label"
key to the desired text value.
with Diagram("\nCluster to Cluster Edges", show=False, direction="TB", graph_attr=graph_attr) as diag:
with Cluster("0", graph_attr={"label":"More meaningful label"}):
a = Node("a", **node_attr)
b = Node("b", **node_attr)
c = Node("c", **node_attr)
d = Node("d", **node_attr)
with Cluster("1", graph_attr={"label":"This is a longer label"}):
e = Node("e", **node_attr)
f = Node("f", **node_attr)
g = Node("g", **node_attr)
h = Node("h", **node_attr)
a >> [b, c]
[b, c] >> d
e >> [f, g]
b >> Edge(lhead="cluster_1") >> f
c >> Edge(ltail="cluster_0", lhead="cluster_1") >> g
c >> Edge(ltail="cluster_0") >> e
d >> [e, h]
diag
Sorry, does workaround still work? I've tried this as an experiment and I'm not seeing the arrows stop at the cluster edges:
with Diagram("Cluster Arrows", show=False):
with Cluster("1a"):
pub1a = PublicSubnet("1b")
with Cluster("1b"):
pub1b = PublicSubnet("1a")
tgw = TransitGateway("tgw")
tgw >> Edge(lhead='cluster_1a') >> pub1a
tgw >> Edge(lhead='cluster_1b') >> pub1b
@geoffreywiseman
You need to include the graph_attr
dictionary, and set the "compound"
key to "true"
.
See https://graphviz.gitlab.io/doc/info/attrs.html#d:compound
compound : bool, default: false If true, allow edges between clusters. See lhead and ltail below. Valid for: Graphs. Note: dot only
Example:
from diagrams import Diagram, Cluster, Edge, Node
from diagrams.aws.network import PublicSubnet, TransitGateway
graph_attr = {
"layout":"dot",
"compound":"true",
}
with Diagram("Cluster Arrows", show=False, graph_attr=graph_attr) as diag:
with Cluster("1a"):
pub1a = PublicSubnet("1b")
with Cluster("1b"):
pub1b = PublicSubnet("1a")
tgw = TransitGateway("tgw")
tgw >> Edge(lhead='cluster_1a') >> pub1a
tgw >> Edge(lhead='cluster_1b') >> pub1b
diag
Ah, yes, I tried reproducing most of what you had, but the compound part I guess I didn't try (or failed to see that it worked). So that helps but not if I want two-way arrows:
graph_attr = {
"layout":"dot",
"compound":"true",
}
with Diagram("Cluster Arrows", show=False, direction='TB', graph_attr=graph_attr):
with Cluster("vpc"):
with Cluster("1a"):
pub1a = PublicSubnet("pub")
priv1a = PrivateSubnet("priv")
with Cluster("1b"):
pub1b = PublicSubnet("pub")
priv1b = PrivateSubnet("priv")
tgw = TransitGateway("tgw")
tgw << Edge(lhead='cluster_1a') >> priv1a
tgw >> Edge(lhead='cluster_1b') << pub1b
@geoffreywiseman
You'll need a Cluster
for the ltail
s as well. Set the Cluster
s graph_attr
as below if you don't want to see it.
from diagrams import Diagram, Cluster, Edge, Node
from diagrams.aws.network import PublicSubnet, TransitGateway
graph_attr = {
"layout":"dot",
"compound":"true",
}
tgw_clus_attr = {
"label":"",
"bgcolor":"transparent",
"penwidth":"0.0"
}
with Diagram("Cluster Arrows", show=False, direction="TB", graph_attr=graph_attr) as diag:
with Cluster("vpc"):
with Cluster("1a"):
pub1a = PublicSubnet("1b")
with Cluster("1b"):
pub1b = PublicSubnet("1a")
with Cluster("tgw", graph_attr=tgw_clus_attr):
tgw = TransitGateway("tgw")
tgw << Edge(ltail="cluster_tgw", lhead='cluster_1a', minlen="2") >> pub1a
tgw << Edge(ltail="cluster_tgw", lhead='cluster_1b', minlen="2") >> pub1b
diag
407 does that nativelly, please check. Nodes can behave as Clusters.
Yes, #407 definitely looks like it'll do what I want in what seems like a simpler way; looking forward to that making it into a release. ;)
This is technically supported in Graphviz, looks like there might already be a PR for this https://github.com/mingrammer/diagrams/pull/439 which replaces https://github.com/mingrammer/diagrams/pull/407
@mingrammer What is the current state of this ticket?
I want to have arrows from and to Clusters, e.g.
All of them throw an exception right now. Is this use case supported or planned? Is there a workaround?