mingrammer / diagrams

:art: Diagram as Code for prototyping cloud system architectures
https://diagrams.mingrammer.com
MIT License
35.72k stars 2.32k forks source link

Merge edges #358

Open servo1x opened 3 years ago

servo1x commented 3 years ago

On very large diagrams, having separate edges for each relationship becomes really noisy. Is it possible to merge edges?

clayms commented 3 years ago

Not exactly sure what your situation is, but using the Advanced Web Service with On-Premise example as a starting point, as it does have a noisy amount of edges.

One hack is to get creative with Custom to create blank placeholders, together with named nodes within Clusters, and then only pointing to single named elements within those Clusters.

Compare the output below to the example output linked to above .

from diagrams import Cluster, Diagram
from diagrams.onprem.analytics import Spark
from diagrams.onprem.compute import Server
from diagrams.onprem.database import PostgreSQL
from diagrams.onprem.inmemory import Redis
from diagrams.onprem.logging import Fluentd
from diagrams.onprem.monitoring import Grafana, Prometheus
from diagrams.onprem.network import Nginx
from diagrams.onprem.queue import Kafka
from diagrams.custom import Custom

with Diagram("\nAdvanced Web Service with On-Premise", show=False) as diag:
    ingress = Nginx("ingress")

    with Cluster("Service Cluster"):
            serv1 = Server("grpc1")
            serv2 = Server("grpc2")
            serv3 = Server("grpc3")

    with Cluster(""):
        blankHA = Custom("","tranparent.png")

        metrics = Prometheus("metric")
        metrics << Grafana("monitoring")

        aggregator = Fluentd("logging")
        blankHA >> aggregator >> Kafka("stream") >> Spark("analytics")

        with Cluster("Database HA"):
            master = PostgreSQL("users")
            master - PostgreSQL("slave") << metrics
            blankHA >> master

        with Cluster("Sessions HA"):
            master = Redis("session")
            master - Redis("replica") << metrics
            blankHA >> master

    ingress >> serv2 >> blankHA

diag

image

clayms commented 3 years ago

Here is another option:

with Diagram("\nAdvanced Web Service with On-Premise", show=False, graph_attr=graph_attr) as diag:
    ingress = Nginx("ingress")

    with Cluster("Service Cluster"):
            serv1 = Server("grpc1")
            serv2 = Server("grpc2")
            serv3 = Server("grpc3")

    (
        serv1 - Edge(minlen="0") - 
        serv2 - Edge(minlen="0") - 
        serv3
    )

    with Cluster(""):        
        metrics = Prometheus("metric")
        metrics << Grafana("monitoring")

        with Cluster("Sessions HA"):
            master = Redis("session")
            master - Redis("replica") << metrics

        with Cluster("Database HA"):
            users = PostgreSQL("users")
            users - PostgreSQL("slave") << metrics

        aggregator = Fluentd("logging")
        aggregator >> Kafka("stream") >> Spark("analytics")

    ingress >> serv2 >> Edge(minlen="2") >> [master, users, aggregator]

diag

image

clayms commented 3 years ago

@servo1x yet another option is to set the graph_attr dictionary key "concentrate" to "true".

Note the following restrictions:

  1. the Edge must end at the same headport
  2. this only works when the minlen of the Edges to be greater than "1".
  3. I could only get this to work when the "splines" graph_attr key was set to the value "spline". It had no effect when the value was set to "ortho", which is the default for the diagrams library.
  4. this will only work with the "dot" layout engine, which is the default for the diagrams library.

For more information see:

  1. https://graphviz.gitlab.io/doc/info/attrs.html#d:concentrate
  2. https://www.graphviz.org/pdf/dotguide.pdf Section 3.3 Concentrators

Here is a simple example:

graph_attr = {
    "concentrate":"true",
    "splines":"spline",
}

edge_attr = {
    "minlen":"3",
}

with Diagram("", show=False, 
        graph_attr=graph_attr,
        edge_attr=edge_attr) as diag:

    with Cluster("Service Cluster"):
        grpsrv = [
            Server("grpc1"),
            Server("grpc2"),
            Server("grpc3")]

    db = PostgreSQL("users")

    grpsrv[0] >> Edge(tailport="se", headport="w") >> db
    grpsrv[1] >> Edge(tailport="e", headport="w") >> db
    grpsrv[2] >> Edge(tailport="ne", headport="w") >> db

diag

image

Here is the same example from the previous posts, but as you can see it is still a bit messy.

graph_attr = {
    "layout":"dot",
    "concentrate":"true",
    "splines":"spline",
}

edge_attr = {
    "minlen":"2",
}

with Diagram("\n\nAdvanced Web Service with On-Premise", show=False, 
        graph_attr=graph_attr,
        edge_attr=edge_attr) as diag:
    ingress = Nginx("ingress")

    metrics = Prometheus("metric")
    metrics << Grafana("monitoring")

    with Cluster("Service Cluster"):
        grpsrv = [
            Server("grpc1"),
            Server("grpc2"),
            Server("grpc3")]

    with Cluster("Sessions HA"):
        sess = Redis("session")
        sess - Redis("replica") << metrics

    with Cluster("Database HA"):
        db = PostgreSQL("users")
        db - PostgreSQL("slave") << metrics

    aggregator = Fluentd("logging")
    aggregator >> Kafka("stream") >> Spark("analytics")

    ingress >> [grpsrv[0], grpsrv[1], grpsrv[2],]
    [grpsrv[0], grpsrv[1], grpsrv[2],] >> Edge(headport="w", minlen="3") >> sess
    [grpsrv[0], grpsrv[1], grpsrv[2],] >> Edge(headport="w", minlen="3") >> db
    [grpsrv[0], grpsrv[1], grpsrv[2],] >> Edge(headport="w", minlen="3") >> aggregator

diag

image

clayms commented 3 years ago

Kept trying and found something better:

with Diagram("\n\nAdvanced Web Service with On-Premise", show=False, 
        graph_attr=graph_attr,
        edge_attr=edge_attr) as diag:
    ingress = Nginx("ingress")

    metrics = Prometheus("metric")
    metrics << Edge(minlen="0") << Grafana("monitoring")

    with Cluster("Service Cluster"):
        grpsrv = [
            Server("grpc1"),
            Server("grpc2"),
            Server("grpc3")]

    blank = Node("", shape="plaintext", height="0.0", width="0.0")

    with Cluster("Sessions HA"):
        sess = Redis("session")
        sess - Redis("replica") << metrics

    with Cluster("Database HA"):
        db = PostgreSQL("users")
        db - PostgreSQL("slave") << metrics

    aggregator = Fluentd("logging")
    aggregator >> Kafka("stream") >> Spark("analytics")

    ingress >> [grpsrv[0], grpsrv[1], grpsrv[2],]
    [grpsrv[0], grpsrv[1], grpsrv[2],] - Edge(headport="w", minlen="1") - blank
    blank >> Edge(headport="w", minlen="2") >> [sess, db, aggregator]

diag

image