All downsampled indices will contain fields of type aggregate_metric_double, which is the result of downsampling numeric fields with time_series_metric attribute set to gauge. Supporting aggregate_metric_double is required to supported tsdb in es|ql.
ES|QL has a much more advanced type system than aggregations. Aggregations almost always return numeric responses as double, whereas es|ql returns numeric fields in the type that they are mapped. This means aggregate_metric_double field type isn't sufficient as it would only allow to query rational gauges that have downsampled and not whole number gauges that have been downsampled.
The aggregate_metric_double field type was sufficient for aggregations, but with es|ql's more sophisticated type system it isn't enough. Therefor we should introduce aggregate_metric_long, which is similar to aggregate_metric_double but for gauges that are whole numbers.
Additionally es|ql needs to understand to the gauge fields are typed differently depending on whether a backing index has been downsampled. For example the first few indices a gauge may be mapped as double, but in the last backing index it is mapped as aggregate_metric_double. We should allow queries to hit all these indices with failing with validation errors.
Tasks:
[ ] Teach es|ql how to read and use fields of the aggregate_metric_double. We should be able to support this field type by letting functions choose which sub field of an aggregate_metric_double field to use as part of the query planning. If no sub field is choses the default sub field is used. #109883
[ ] Allow the same fields to have different types in different indices. E.g. a field may be mapped as aggregate_metric_double and double (and smaller type floating point types) in es|ql.
[ ] Introduce a aggregate_metric_long field type and teach downsampling to use that instead of aggregate_metric_double in cases the original field is a whole number like field type.
[ ] Add support for aggregate_metric_long field type in es|ql.
All downsampled indices will contain fields of type
aggregate_metric_double
, which is the result of downsampling numeric fields withtime_series_metric
attribute set togauge
. Supportingaggregate_metric_double
is required to supported tsdb in es|ql.ES|QL has a much more advanced type system than aggregations. Aggregations almost always return numeric responses as double, whereas es|ql returns numeric fields in the type that they are mapped. This means
aggregate_metric_double
field type isn't sufficient as it would only allow to query rational gauges that have downsampled and not whole number gauges that have been downsampled.The
aggregate_metric_double
field type was sufficient for aggregations, but with es|ql's more sophisticated type system it isn't enough. Therefor we should introduceaggregate_metric_long
, which is similar toaggregate_metric_double
but for gauges that are whole numbers.Additionally es|ql needs to understand to the gauge fields are typed differently depending on whether a backing index has been downsampled. For example the first few indices a gauge may be mapped as
double
, but in the last backing index it is mapped asaggregate_metric_double
. We should allow queries to hit all these indices with failing with validation errors.Tasks:
aggregate_metric_double
. We should be able to support this field type by letting functions choose which sub field of an aggregate_metric_double field to use as part of the query planning. If no sub field is choses the default sub field is used. #109883aggregate_metric_double
anddouble
(and smaller type floating point types) in es|ql.aggregate_metric_long
field type and teach downsampling to use that instead ofaggregate_metric_double
in cases the original field is a whole number like field type.aggregate_metric_long
field type in es|ql.