Is it possible to create a global diagram object for various graph workflow rendering?

[sheldonhull]

Prior Work: #124

• I want to create a global diagram object
• Then use the functions below to allow me to generate a new "path" for the lines without me needing to redeclare all the diagram objects each time.

ℹ️ flow function

``` def colored_flow(nodes, color="black", style="", label=""): print(f"==> def colored_flow: nodes {nodes}") for index, n2 in zip(nodes, nodes[1:]): print(f"\t----> {index.label} >> Edge(label={label}) >> {n2.label}") index >> Edge(label=label) >> Edge(color=color, style=style) >> n2 print("completed with colored_flow") ```

ℹ️ creating a diagram object

``` mainsourcefile = os.path.join(artifact_directory, "main") main_gv = f"{mainsourcefile}.gv" with Diagram( name="main", direction=direction, graph_attr=graph_attr, show=show, filename=mainsourcefile, outformat=outformat, ) as diag: with Cluster("vpc"): DataDog("", width=smaller, height=smaller) igw_gateway = InternetGateway("internet-gateway") with Cluster("subnets-public"): nat_gateway = NATGateway("nat") alb_app1 = ALB("alb1l") alb_app2 = ALB("alb2") with Cluster("subnets-private"): with Cluster("ECS Task 1"): container_1 = ECS("container_1") container_2 = ECS("container_2", width=smaller, height=smaller) diag diag.dot.save(filename=f"{mainsourcefile}.gv") ```

I'm trying to use this object but can't figure out how to keep the diagram variable in the current context. I've tried using contextvars and overriding, and more, but I'm not a Python dev so some of those nuances are a bit unclear to me.

ℹ️ GenerateDiagram Function

``` def GenerateDiagram( title, direction="LR", graph_attr=graph_attr, show=show, filename=filename, outformat=outformat, flow=[], ): print(f"============= {title} ============= ") print(f"title : {title}") print(f"flow : {flow}") print(f"filename : {filename}") with Diagram("", show=False, ) as out_diag: pass out_diag.subgraph(diag_import_main) <<<<<<<<------ TRIED THIS BUT DOESN'T HELP colored_flow( nodes=(flow), color="darkblue", style="solid,setlinewidth(5)", ) out_diag ```

Ok, now here's where this breaks down. The nodes to pass in the flow are objects in the context of a diagram and able to be parsed as nodes. However, without this Diagram context, they are just strings and the objects aren't recognized to map. Importing with Diagraph loses these node properties as well.

ℹ️ Attempt to Generate Diagram Flow With Function

``` diag_import_body1 = Source.from_file(filename=main_gv,format="gv").source.split('\n')[ 1:-2 ] global diag_import_main diag_import_main = Digraph(body=diag_import_body1) filename = os.path.join(artifact_directory,'another_workflow') GenerateDiagram( title="another_workflow", direction="LR", graph_attr=graph_attr, show=show, filename=filename, outformat=outformat, flow=( igw_gateway,alb_app1,container_1 ) <<<<<<<<<<<<<<--- not in scope ) ```

What I'd really like to do is create this diagram object to persist and add flow information and edit it to rerender with different changes.

Any tips?

[clayms]

I think the no-op flag could do this. However, it needs to be implemented in the underlying graphviz python library first https://github.com/xflr6/graphviz/issues/131

Have a look at the following comments. https://github.com/mingrammer/diagrams/issues/436#issuecomment-758717910 https://github.com/mingrammer/diagrams/issues/436#issuecomment-800563909 https://github.com/mingrammer/diagrams/issues/487#issuecomment-800517538

[sheldonhull]

For reference, I did try that, but the ability to nest another diagram wasn't specifically what I was trying to do. Once it's reloaded I'm not seeing how I can then reference the various objects by name to build a flow.

Without the ability to set the diagram object as a "global" object I can then pass the desired "flow" into, I'm stuck with a diagram that isn't able to be customized with edge to edge definitions. Does that make it more clear?

[clayms]

If I understand you correctly, the only option that comes close to what you want to do would be to use the no-op flag, which is not implemented yet in the underlying graphviz python library. Therefore, you will have to use the dot language directly, which is definitely doable. Even then, you will have to manually choose the positions (x,y) of the nodes and then load the graph and connect with Edges how you want.

If what you want is to keep a diagram "object" with all of the nodes and no layout information at all and then only specify different "flows" that dictate how the nodes will then be connected, I don't think that is possible.
Perhaps try and make a diagram with no edges, save it, load it, then specify the edges.

[sheldonhull]

@clayms thank you! I already did the graph creation but upon loading it I'm not sure how to specify the edges again as it's now a graphviz object not a "diagram" object.

If I can load the dot file and the specific edges that would work too, but not sure if that's achievable

[clayms]

Try and modify the graphviz list of body elements directly after you reload your graphviz file.

Print out a the body elements of a re-loaded graphviz file that has Edges. You will see that the Edges are just nodeid_1 -> nodeid_2 followed by a list of Edge attributes.
You can simply append the Source list of body elements with the edges you want.

The tricky part will be to identify which node_id belongs to which node_label.

Where diag.gv is your saved dot graphviz file.

from graphviz import Digraph, Source

diag_import_body = Source.from_file(filename="diag.gv", format="png").source.split('\n')[1:-2]

## inspect
for s in diag_import_body: 
    print(s)

[clayms]

@sheldonhull Try the following:

create and save diagram with no Edges

from diagrams import Diagram, Cluster
from diagrams.aws.compute import EC2
from diagrams.aws.database import RDS
from diagrams.aws.network import ELB

with Diagram("Grouped Workers", show=False, direction="TB") as diag_2:
    elb = ELB("lb") 
    ec2 = EC2("worker1")
    rds = RDS("events")

diag_2.dot.save("diag_2.gv")

Load saved diagram with no Edges and view.

from graphviz import Digraph, Source

diag_import_body_nodes = Source.from_file(filename="diag_2.gv", format="png").source.split('\n')[1:-2]

diag_import = Digraph(body=diag_import_body_nodes)

with Diagram("", show=False, ) as out_diag:
    out_diag.subgraph(diag_import)

out_diag

Create dictionary of Node labels : Node IDs

node_label_id_list = [s.replace("\t","").replace("[label=", "").split()[0:2][::-1] for s in diag_import_body_nodes if "image=" in s]
node_label_id_dict = {l[0]:l[1] for l in node_label_id_list}

Create Edges ("flow") between chosen Nodes and append to list of graphviz Source body elements.

edge_attr = ' [dir=forward fontcolor="#2D3436" fontname="Sans-Serif" fontsize=13]'
Edge1 = ["\t" + " -> ".join([node_label_id_dict.get("lb"), node_label_id_dict.get("worker1")]) + edge_attr]
diag_import_body_flow1 = diag_import_body_nodes + Edge1

Create diagram from appended list of graphviz Source body elements and view.

diag_import = Digraph(body=diag_import_body_flow1)

with Diagram("", show=False, ) as out_diag:
    out_diag.subgraph(diag_import)

out_diag

[sheldonhull]

Wow! Very helpful. This definitely gets complicated once it's used beyond the standard design, so will have to try this out when I can. I recently blogged on your project fyi, as I had a great time using to visualize some AWS architecture, so thanks for this!

Diagrams as Code

I think doing graphviz manipulation extensively defeats the design of this library which is to provide a clean intuitive experience, so while I might try this out, if it's not enhanced in the library itself, I'm not sure I'll force this.

[sheldonhull]

One last tip as a newer user... I'd suggest these examples you provide be added to a "rough examples" doc and not sit in the issues themselves. I found a lot of useful tips from you in the issues, but had to dig a ton.

Would love to see these captured in a less formal doc for your site to give edge case examples so this great work you've done in investigation doesn't get lost!