search

inducer / pytato

Lazily evaluated arrays in Python
Other
8 stars 16 forks source link

Visualization improvements #526

Open majosm opened 1 month ago

majosm commented 1 month ago

I just wanted to brainstorm some ideas here for improving visualization of large DAGs, with the intent of possibly finding one or more improvements that @kajalpatelinfo could work on while she's still with us for the summer.

I don't know if there have been any past discussions about this; at least I didn't see anything while browsing through past issues. I have a couple of vague ideas for improvements that I think would be useful. However, I'm not very familiar with the visualization code, so I don't have a sense of how difficult they would be to implement. Here they are in any case:

  1. Collapsing subexpressions. I'm picturing tagging a node that represents the result of some calculation (temperature, RHS, whatever), and then collapsing all of the nodes that go into that calculation into one visualization node. (How it would decide what exactly to collapse, I'm not 100% sure. Maybe something along the lines of collapsing a node if all of the nodes that depend on it are also part of the collapsed set? Might require a different traversal order than what we usually do with mappers.)
  2. Coloring nodes. If one wants to find a certain node in the DAG easily, they could tag it to be colored a certain way just before calling show_dot_graph.

Any thoughts @inducer, @kaushikcfd? I'd be interested in hearing about any other visualization ideas you've thought of as well.

majosm commented 1 month ago

@matthiasdiener too 🙂

matthiasdiener commented 1 month ago

I remember that @kaushikcfd had code that would only show the type of node (with different node colors and shapes), without the metadata like id, tags, etc. I think this can be helpful when visualizing large DAGs.

inducer commented 1 month ago

My main problem with the current visualization is that it's incredibly verbose. IMO, ideally, a node adding two arrays would look like this:

grafik

with everything else in a tooltip that can pop up on request. (That is something that graphviz supports.)

nkoskelo commented 1 month ago

In using show_dot_graph, I notice that there are times that the information about an individual operation can sometimes extend over the bounding box drawn for the operation. I like @inducer's request to make the information appear as a pop up.