Open RussTedrake opened 4 years ago
I spent a little more time looking through the options yesterday, but haven't found one that is obviously good.
To address #13975, I realized pydot could create svg. For moderate size diagrams, that is a reasonable solution for diagram visualization, too.
I went through the entire list of javascript packages in https://codefreezr.github.io/awesome-graphviz/#language-bindings , as recommended by @jwnimmer-tri in #13705. Those links took me to https://github.com/dagrejs, which is the most promising path I've seen from graphviz to a javascript rendering. But even this package doesn't appear to support collapsing/expanding subdiagrams, so I feel the additional value it adds over the python-based svg rendering is not worth the trouble.
Because of the existing SVG solution, I am going to upgrade the feature request of collapsing/expanding here to a "must have". I honestly haven't found a good gojs alternative. I'm almost of the opinion that we should just write our own in d3. I bet I could do some pretty nice graph layout optimization in mathematicalprogram (probably a MILP).
I'm also ok if it takes a different form from collapse/expand. It might be possible to use a combination of max_depth
on the graphviz and links in something like dagrejs to open subdiagrams.
Seems like someone with js chops (not me) could do something decent with https://github.com/magjac/d3-graphviz (BSD 3-clause license). If it is possible to enumerate the set of expanded/collapsed graphs ahead of time, it may not be too hard to provide some navigation among them.
Does @trowell-tri fit that bill?
There is one deprecated example of using GenerateHtml
left in https://deepnote.com/project/Ch2-Lets-get-you-a-robot-j4YXK7WXTOubrZLRGsemzA/%2Frobot.ipynb which shows the old feature that we are trying to replace.
The recent https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/ caught my eye. Possibly its https://github.com/mermaid-js/mermaid#readme will be a winner in terms of a durable graph modeling language for us to build upon.
mermaid looks great, but it doesn't look to support collapsable subdiagrams. I think that's a required feature for this.
I was just searching for interactive (i.e. collapsable) mindmap/diagram libraries myself and came across this issue. https://github.com/gera2ld/markmap (their demo) fit my bill. Not too sure about the specific needs here, but sharing fwiw.
Chatted with Russ a bit yesterday and I understand the needs better now. https://js.cytoscape.org/ and its plugin https://ivis-at-bilkent.github.io/cytoscape.js-expand-collapse/ looked promising.
Building on top of those (and some other auxiliary libs, all MIT license), I cobbled together https://shensquared.github.io/diagram-proto/demo.html. The interaction isn't intuitive yet, so here's a gif.
Comparing with the one in https://manipulation.csail.mit.edu/intro.html#example3 that Russ made with GoJS, I think we'd still need:
Both feel doable. I think I actually know how to tweak the listener/event. For the visual, cloning Russ's graph is for sure possible; in fact, Cytoscape's nodes/edges API looks pretty similar to GoJS (JSON based, uses keys like from, to, group, etc). I do need to look closer if their layout also has a smart ‘auto’ mode like GoJS, but even if it's not built in, seeing that they have a lively community (building extensions, plugins, etc), I’m optimistic.
Before we pursue this further though: wonder if I missed anything on the wish list?
Thanks @shensquared ! I'm not so concerned about how "polished" the collapse/expand looks; as long as you can connect multiple ports in one diagram and get somewhat reasonable layouts, then this looks great!
Sounds good. My to-do plan then, in order:
Generating
function (i.e. takes a .yaml
or a python dictionary and pops out the .html
automatically)Will let you know if/when the first two (the essential ones) are done.
@RussTedrake Updated the demo towards the first 2 items on my list. Does it look like what you had in mind?
In parallel, I plan to start trying auto-generate a real Drake diagram (instead of using 'Lorem ipsum' nodes/edges). Maybe from e.g. a .yaml
? Wonder how/where did you get the port names and connections for your GoJS diagram?
Looks great!
On Tue, Mar 15, 2022 at 10:26 PM, Shen Shen @.***> wrote:
@RussTedrake https://github.com/RussTedrake Updated the demo https://shensquared.github.io/diagram-proto/demo.html towards the first 2 items on my list.
— Reply to this email directly, view it on GitHub https://github.com/RobotLocomotion/drake/issues/13874#issuecomment-1068666089, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRE2NAYFNALL3Y53SNLCMDVAFBETANCNFSM4QAFRCSA . You are receiving this because you were mentioned.Message ID: @.***>
Sorry it's been a while.
Below is a mimic of Russ's manip_station diagram: The demo; it's based on Cytoscape.js, and line 24-34 toggles different edge styles and layouts.
Some shortcomings when compared with gojs:
As I was trying to improve these, questions/answers like this makes me a bit less optimistic about the base library.
So while I'm still trying, I'm also keeping an eye for other libraries. I'm also happy to bring this one in as is into GenerateHTML
first (e.g. if we wanna first see how other Drake diagrams might look like with this?)
@RussTedrake any preference? thanks!
The demo looks/works a lot better to me now.
The rendering is via Cytoscape; the collapse/expand is via cytoscape.js-expand-collapse; the layout is done by elk and then wrapped/parsed via cytoscape.js-elk. While not perfect, this combo is my favorite so far :D
Fwiw, other libraries that didn't quite work out but probably would be great depending on the need (so 'good-to-know's):
That demo does look great. Thanks for doing it! Is there any chance that the input ports / output ports for a single system (e.g. the plant in this demo diagram) can be left/right justified?
seems possible in elk though the wrapper has some lost-in-translation; I'll look more into it.
Is there any chance the input ports / output ports for a single system (e.g. the plant in this demo diagram) can be left/right justified?
Check this out?
Also added in that demo:
Feel free to let me know too if these are irrelevant/counterproductive? Thanks.
I think the WebGME is a good choice, it can achieve the rendering of system diagrams and drag/drop modeling. But unfortunately, I'm not good at coding. Can anyone achieve it? @RussTedrake @shensquared @rpoyner-tri @jwnimmer-tri
(I wrote this in an email for someone potentially willing to help with the cause; but realized it deserves to be here in an issue for anybody that might want to help).
Quick background:
Drake builds system diagrams like Simulink, etc (but programmatically in python/c++, instead of visually). Example here: https://mybinder.org/v2/gh/RobotLocomotion/drake/nightly-release?filepath=tutorials/dynamical_systems.ipynb
You’ll see in the second-to-last cell on that tutorial that, because of it’s c++ origins, we never spent much time visualizing the resulting diagrams. But we offer the ability to kick-out a graphviz .dot file. In jupyter notebooks, we render that to a PNG and display the PNG.
I took a spin earlier this summer and made a version that renders to a js diagramming tool, instead: https://github.com/RobotLocomotion/drake/pull/13644 i did the javascript coding for this in https://observablehq.com/@russtedrake/gojs-for-drake-system-diagrams
It turns out the particular diagramming tool that i choose was rejected because of the licensing implications: https://github.com/RobotLocomotion/drake/issues/13705 but, as you see at the bottom of that issue, it might be possible to simply render the .dot file in js.
I think a few hours of trying some of the js diagramming libraries and deciding what works would result in an immense improvement in the user/student experience. I think one key feature is the ability to expand/collapse nested subdiagrams.