influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
29.07k stars 3.56k forks source link

Continuous queries are time dependent(?) #17169

Open pohmelie opened 4 years ago

pohmelie commented 4 years ago

Hi there! We use influxdb for storing trade candles and aggregate them by hour and day interval. We have such cq:

day  CREATE CONTINUOUS QUERY day ON candles BEGIN SELECT first(open) AS open, max(high) AS high, min(low) AS low, last(close) AS close, sum(base_volume) AS base_volume, sum(quote_volume) AS quote_volume INTO candles.day.:MEASUREMENT FROM candles.minute./.*/ GROUP BY time(1d) END

Keypoint is there are last, first, max and min aggregation. But, resulting data is non-consistent:

> select * from candles.day.binance_DOGEUSDT order by time desc limit 2;
name: binance_DOGEUSDT
time                base_volume close     high      low       open      quote_volume
----                ----------- -----     ----      ---       ----      ------------
1583712000000000000 163524475   0.0022181 0.0022167 0.00208   0.0021714 352818.05889130005
1583625600000000000 136738799   0.0021792 0.0024039 0.0021651 0.0024029 311099.2201635999

You can see that close is bigger than high. Maybe there should be some tricks with cq to avoid such behavior?

romantolkachyov commented 4 years ago

InfluxDB v1.7

pohmelie commented 4 years ago

Okay, probably the problem is with «during continuous query we add some data» we can solve this with resample, but is it written somewhere in docs, that queries are not atomic?