search

grafana / iot-sitewise-datasource

IoT Sitewise
Apache License 2.0
19 stars 9 forks source link

Concurrent fetching and caching of data #262

Open mitchlee-amzn opened 7 months ago

mitchlee-amzn commented 7 months ago

What this PR does / why we need it: Loading a huge amount of data takes a lot of time and also, if the dashboard is set to auto refresh then most of the times the dashboard refreshes while the data is still getting loaded. This is not a good user experience and makes the dashboard unusable. The changes included fixes the data loading issue by implementing the concurrent fetching of data for Get property value history query and also by supporting the caching logic.

sarahzinger commented 6 months ago

Hey Folks,

thanks for taking a look at this, and bringing these performance issues to our attention! I think concurrent fetching and caching of data might add a fair bit of complexity to our system and might be a fairly large change with potentially unintended consequences.

I'm wondering if we can first see if this pr helps improve performance enough so that it's not an issue. We also are currently working to make this plugin streaming-capable the way twinmaker is, which I think might also be a better fit for this user. Perhaps we can revisit after those two prs are merged and if we still see performance issues we then consider adding in a frontend cache?