search

grafana / iot-sitewise-datasource

IoT Sitewise
Apache License 2.0
19 stars 9 forks source link

WIP backend streaming #277

Open sarahzinger opened 6 months ago

sarahzinger commented 6 months ago

I've been investigating how we might implement streaming for this datasource plugin, and it's become a bit larger/more confusing than I expected so sharing a not-quite-working WIP for now. (feel free to ignore git commits, I tried a few different methods to getting here and most of them are not useful)

Here's a flow chart of how I'm thinking this could work: image

I still need to learn more about the relationship between Grafana Live and streaming, I mostly based this work off of what I've seen us do with twinmaker, but I also took a look at https://github.com/grafana/grafana-plugin-examples/tree/main/examples/datasource-streaming-backend-websocket

In it's current state I have more than a few bugs that I'm trying to track down (for example when I switch to streaming, I get the wrong data back sometimes?) So if you see where I've gone wrong, feedback is very appreciated!

I also wonder at a higher level, if any of this is necessary, it looks like we have attempted to implement streaming on the frontend in a way? and I while I do think that communication over websockets is generally faster than making http requests repeatedly, but I wonder if the problems users with large amounts of data are seeing will be solved by this? Interested if others have thoughts with the overall approach.

chejimmy commented 2 months ago

I'm interested in learning more about this approach, if you can help me understand the followings 🙏

  1. how does streaming help with performance?
  2. would streaming improve other quality of the plugin? e.g. cost saving as a side product of the storing SiteWise responses