Graphical view of ERSAP/RTDP configuration

[faustus123]

We need a way to visualize an RTDP configuration. This should include visualizations of relevant telemetry from the individual components that updates as the platform runs the simulation/workflow. For example, if an FPGA component is in the configuration, values for data in/out rates, utilization, etc..

A web interface might be nice, but a native linux app that can be run in a container with GUI support would work (e.g. X11).

[gurjyan]

Several packages were evaluated for the purpose of composing ERSAP applications, specifically focusing on a dataflow graphical editor and visualization tools. These options ranged from CODA JCEdit to BPMN (business process modeling notation) editors. My investigation led me to examine the open-source COMUNDA framework.

[faustus123]

I found a Camunda framework (https://camunda.com/) that looks related. The open source part on github is here: https://github.com/camunda .

This looks like a lot more than just a visualization tool. I think you're right that this warrants a deeper look to see how much of it we can leverage.

[cissieAB]

Interactive framework:

• Dash: https://dash.plotly.com/, examples: https://dash-example-index.herokuapp.com/

Visualization

• Plotly (work with Dash), gallery: https://plotly.com/python/,
• Echart. Gallery: https://echarts.apache.org/examples/en/index.html
• Graphviz, Gallery: https://graphviz.org/Gallery/directed/

Chart type: Sankey, DAG, or more flexible?

[gurjyan]

Graphviz aligns more closely with our needs, particularly as a tool for visualizing workflows. Additionally, please explore the following: https://decisions.com/no-code-platform It is crucial to obtain an XML or an alternative programmatic depiction of the workflow's directed graph, which can then be transformed into ERSAP YAML or CODA COOL format.

[cissieAB]

Advised by ChatGPT, I utilized the [dash_cytoscape](https://dash.plotly.com/cytoscape) library to implement the above PRs.

More feutures needs to be implemented:
- [ ] Configurable edge width based on some backend data.
- [ ] "Real-time" Prometheus metrics display on the frontend UI.

dash_cytoscape is going to be taken over by the Grafana.

[cissieAB]

Grafana "node-graph" visualization:

• Doc: https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/node-graph
• Node-graph API: https://github.com/hoptical/nodegraph-api-plugin, including guides on how to install and configure the backend data for this visualization.

[cissieAB]

@faustus123 @gurjyan I found the nodegraph-api-plugin and was able to got a first-step demo based on the HelloWorld configuration. Use a Python code to start a Flask backend at localhost:5000 and generate some data in JSON. After installing the Grafana plugin, tell Grafana to use the data source from localhost:5000, it will displaythe above visualization. "thickness" is a field of the "edges". I set one of them "5" and the other "1", so they look a little different.

Commit is at: https://github.com/JeffersonLab/SRO-RTDP/commit/b32a5f57fb6140022314b1f0b6bce0824848d7d8