search

google-ai-edge / model-explorer

A modern model graph visualizer and debugger
Apache License 2.0
971 stars 70 forks source link

[Feature Request] Allow cycles between nodes #65

Open el-agua opened 3 months ago

el-agua commented 3 months ago

Often times, it may be useful for cycles to be displayed in the graph between individual nodes. Right now, this is impossible.

Example of a control feedback loop

image

Currently, if I tried to construct this graph with individual nodes. You get the following:

image

However, if I create an "in" and "out" node, representing each of these functions as layers, I get the desired behavior.

image

However, this can be quite tedious. Might be useful as a toggle in the settings?

pkgoogle commented 3 months ago

Hi @el-agua, can you help us out and perhaps share model(s) or code to reproduce those models which display both behaviors? Thanks.

el-agua commented 3 months ago

https://github.com/el-agua/model-explorer-cycle-example/tree/main

Adapter files

pkgoogle commented 2 months ago

Hi @el-agua, we're trying to figure out what exactly is the use case for this in order for us to prioritize it correctly. Can you maybe describe your exact use case and the type of model you are using and its use?

el-agua commented 2 months ago

I have found model explorer, using the custom adapter api, to be incredibly useful for modeling large control systems. As with most feedback control, cycles are inevitable.

It seems that with the workaround, the frontend can already support such a display. However, direct support would greatly simplify this. Even something along the lines of visualizing RNN/SSM blocks would benefit from this?

pkgoogle commented 2 months ago

Hi @el-agua, since your use case is not specific for ML models, we don't quite have the bandwidth to prioritize this use case. That being said, once the front-end becomes open source we are certainly welcoming contributions to add this. Thanks for your understanding.