rkovachFF
commented
3 years ago I'm running a pilot to test using Timestream and Grafana, and have experienced similar symptoms with the latest version. Each time I run a query, I may get all data returned, only some series, or no data. Based on the behavior, I suspect there is a timeout occurring at some level - it appears that the behavior is not a function of the size or the data within the resultset, as I can run the same query against the same available data over and over and get a range of different results.
In my case I have a TS table with a few thousand meters, and for each meter have several months of readings at 5min intervals. Using this query:
WITH readings_agg AS ( SELECT meterid, measure_name, bin(time, 5m) as time, avg("measure_value::double") as measure_value FROM "MonitoringPOC"."readings_5min" WHERE measure_name = 'readingvalue' AND $__timeFilter AND meterid in ('11','15','16' --,'3','4','17','21','22','23','27','28','29' ) GROUP BY meterid, measure_name, bin(time, 5m) ) SELECT meterid, measure_name , CREATE_TIME_SERIES(time, measure_value) AS readingvalue FROM readings_agg GROUP by measure_name, meterid
if I set the bin interval to 5m and duration of an entire month, I can include at most 1 meterid in my IN clause - any more will result in "No data" (and sometimes even only the 1 yields this result). Setting it to 15min I can often (but not consistently) include a half-dozen meterids. Setting it to 1hr allows me even more meterids. Similarly, if I specify a much shorter duration I can use a more granular interval or more meterids. But the randomness of the behavior suggests that the issue is not the size of the resultset but rather some kind of timeout.
If I run the same queries using the Timestream console, the queries that typically fail (or return only partial results) often show run times of 5s or more, and the ones that work most of the time via the grafana datasource plugin are those that take <3 seconds via the console. In grafana, the dashboard response time to refresh seems to not be significantly longer than the query takes when I run it in the TS console.
As it is behaving right now, this plugin is not really usable by us for 3 reasons: 1) the inconsistency of behavior means that we cannot create reliable workarounds to whatever the existing limitation is, 2) the current behavior of simply failing to return some or all of the data leaves our end user with no way to determine whether there is truly no data in the system for that time period or that they are trying to retrieve too much data, and 3) my baseline minimum expectation is that we should be able to include at least 6 chart series on a line graph with 5min interval data for 30 days (and realistically want that for at least 3 months) - as it is right now I can't even reliably display a single series for 1 month.
ryantxu
I am trying to display a time series with multiple series from a query like this:
I have about ~30 hosts I'm pulling from, but if enough data points are returned by the query (based on the time filter and interval), only the last x number of series are displayed - I assume after some threshold of data points is reached. It would appear to me that because it is the last segment of data that is displayed that the pagination logic is included, but some how the first number of data points are being discarded at some point.