Optimize `keep()/drop() |> aggregateWindow` Flux Queries

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics

https://influxdata.com

Apache License 2.0

28.84k stars 3.55k forks source link

Optimize `keep()/drop() |> aggregateWindow` Flux Queries #22472

Open williamhbaker opened 3 years ago

williamhbaker commented 3 years ago

Flux queries involving keep() |> aggregateWindow or drop() |> aggregateWindow are slower than their InfluxQL equivalents. This issue is to optimize the Flux queries to be on par with InfluxQL.

For keep, reference the community issue here: https://github.com/influxdata/influxdb/issues/18088 For drop, see: https://github.com/influxdata/influxdb-comparisons/pull/198

danxmoran commented 3 years ago

I have a hunch that both keep() |> aggregate and drop() |> aggregate could be rewritten to equivalent group() |> aggregate statements, allowing us to (eventually) share the benefits of #22469. keep feels like a group-by, and drop feels like a group-except. I haven't fully thought through all the edge cases we might hit trying to make the translation...