OpenTSDB / opentsdb

A scalable, distributed Time Series Database.
http://opentsdb.net
GNU Lesser General Public License v2.1
5k stars 1.25k forks source link

Plugin for performing aggregation, rewrite, blacklist/whitelist, downsampling at write time #802

Open johann8384 opened 8 years ago

johann8384 commented 8 years ago

I submitted blacklist/whitelist code and @manolama converted that into a plugin.

I'd like to extend or fork+extend that plugin to support similar actions to Graphite's Carbon Relay.

http://graphite.readthedocs.io/en/latest/config-carbon.html

Downsampling

This one is not quite the same as the Carbon Relay intent, but basically buffer for $period seconds and write out $metric.$label.$aggregationMethod with the new dowsampled value

6 10s interval datapoints for webserver.requests becomes 1 1m interval datapoint for webserver.requests.1m.avg

[1m_avg]
period = seconds
label = 1m
pattern = \webserver.*$
aggregationMethod = avg

Aggregation

output_template (frequency) = method input_pattern
<env>.applications.<app>.all.requests (60) = sum <env>.applications.<app>.*.requests
<env>.applications.<app>.all.latency (60) = avg <env>.applications.<app>.*.latency

Rewrite

regex-pattern = replacement-text
^collectd\.([a-z0-9]+)\. = \1.system.
collectd.prod.cpu-0.idle-time => prod.system.cpu-0.idle-item

Blacklist/Whitelist

Already implemented in 2.3.0

johann8384 commented 7 years ago

@manolama Is there something in the 3.0 refactoring that would make this possible? (I assume the rollups cover part of this). Perhaps we can identify which parts are covered and which parts are still "needs to be considered"