search

GridProtectionAlliance / osisoftpi-grafana

Grafana Data Source Plugin for the OSIsoft PI System
https://grafana.com/plugins/gridprotectionalliance-osisoftpi-datasource
Apache License 2.0
58 stars 30 forks source link

Streaming via PI WebAPI websocket #99

Open txjmb opened 1 year ago

txjmb commented 1 year ago

Does the plugin currently have the ability to stream data via WebAPI Websocket? It would be very valuable to have a streaming connection to get real-time updates from PI server, and could be more timely, efficient, and easier on the server than polling. This would fall into the feature request category if it is not currently supported.

Relevant documentation from Grafana: https://grafana.com/docs/grafana/latest/developers/plugins/build-a-streaming-data-source-plugin/

Documentation on channels can be found in the PI WebAPI built-in documenation at /piwebapi/help/topics/channels.

complacentsee commented 1 year ago

I took a rough look at this (ee7dbda18fd4608ef36d51c6e5fee616b91929bd), and I think the improvements are are not offsetting the significant issues it comes with. The biggest challenge is that it only makes sense with the Access parameter set to "Browser" at the moment. The "Server" option has no way to proxy the websocket connection using your grafana server, and the websocket needs to be opened directly from your browser.

Without trying to speak for the developer team, I think this function would make more sense if the plugin was updated with a backend server component.

complacentsee commented 1 year ago

Actually, I was wrong. You can pass the websocket traffic through the proxy like this: 5c275723d63689d95a62a467b188c43724103cf6.

txjmb commented 1 year ago

Thank you so much for looking into this so quickly. Sounds like maybe there's a way forward on this?

If I can help in any way, please let me know. I can check out your commit and see if I can find any issues.

txjmb commented 1 year ago

One additional thing we would want to do besides connecting the websocket with the "includeIntitialValues" flag is to do a simple query first to determine size of the CircularDataFrame and to fill the CircularDataFrame with back data from the dashboard date range (say, for instance that the dashboard showed the last few hours to now). Then, any additional data that comes in on the websocket would be added to the CircularDataFrame if it falls inside the date range.

I can confirm with the "Live Measurements" Grafana streaming demo data source that the CircularDataFrame is indeed...circular. The data rolls when date range is set to "last x minutes". Makes perfect sense, and shouldn't really confuse users. It's an open question if we would want to have the UI include an option to "Use websocket", or just make it do that automatically, as I can't really think of a downside, unless that feature is explicitely blocked or turned off for a particular user on their WebAPI server . The CircularDataFrame could still be used if they choose not to. If they are querying a range in the past, it will behave just like a regular query, since nothing would come in on the websocket that would fall into the query range and therefore would not be added to the data frame.

This would be a very good add for our industry (power) as some data is updated as often as every 4s or less, and would greatly reduce the load on the WebAPI servers when users want to view real-time data in near real-time (which would, without this feature, mean they would have to query the server every few seconds, with all the round-trip and querying overhead).