OpenTSDB / opentsdb

A scalable, distributed Time Series Database.
http://opentsdb.net
GNU Lesser General Public License v2.1
4.99k stars 1.25k forks source link

Support aggregations for Annotations #469

Open kylemallory opened 9 years ago

kylemallory commented 9 years ago

I would like to see the option to allow the aggregation of annotations, grouping multiple annotations into a single "higher level" annotation.

I'm not sure exactly how this would work with the current annotation engine, but I thought I'd throw this out there anyway. Here is a proposed use case:

We generate annotations for various stages of our deployment process, which can involved the staging and committing of individual components to dozens of components, depending on the nature of the release.

We would like to be able to aggregate each of those component releases into a higher-level "deployment" annotation that spans the timeframe of all the individual releases.

Since the annotations are tied to a TSUID, a query which returns a multiple tsuids will return the annotations associated with each tsuid, individually. Ie, "m=deployment{component=*}" will return all annotations for all components (which is different than "m=deployment").

Since tsuids are tied to metrics & tags, perhaps something like "m=deployment {component=}?groupAnnotations={}" could combine all annotations, regardless of tags, into a single annotation that spans from the start of the first annotation, to the end of the last annotation.

Going a step further, "m=deployment {component=, host=}?groupAnnotations={component}" could combine annotations of differing components, into a single annotation per host.

If this feels a little backwards, its because I chose an explicit aggregation rather than implicit (ie, "splitAnnotations={host}") because that would dictate a default behaviour of combining annotations, changing current behaviour.

To be determined is how names/descriptions/custom data would be combined when aggregating.

Another approach would be the nesting of annotations into a tree, rather than a flat grouping.

manolama commented 9 years ago

It does sound useful but definitely tricky. Could you mock up some example outputs so we can see what you're thinking of a little easier? thanks!