influxdata / kapacitor

Open source framework for processing, monitoring, and alerting on time series data
MIT License
2.32k stars 492 forks source link

{{.Tags}} is empty when querying InfluxDB populated by Collectd #1502

Open yakneens opened 7 years ago

yakneens commented 7 years ago

I'm using a setup where Collectd is pushing metrics into InfluxDB with alerts via Kapacitor and Chronograf on top. Influxdb is version 1.2 and Kapacitor and Chronograf is version 1.3. I can successfully create alerts via TICK files or via Chronograf and they get triggered appropriately, however when I try to format my message I don't seem to be able to get access to the relevant tag_keys and tag_values via the {{.Tags}} map. I use the following message in Chronograf

{{ index .Tags "host" }}:{{ .Name }}:{{ index .Tags "type_instance" }} is {{.Level}} value: {{ index .Fields "value" }} @ {{.Time}} {{.Tags}}

and receive the following message on Slack as a result:

:cpu_value: is CRITICAL value: 47.12757803790413 @ 2017-07-28 13:21:00 +0000 UTC map[]

The map of tags appears empty.

This is the TICK script that gets generated by Chronograf:

`var db = 'metrics'

var rp = 'default'

var measurement = 'cpu_value'

var groupBy = []

var whereFilter = lambda: ("host" == 'salt-master') AND ("type_instance" == 'idle')

var period = 10s

var every = 30s

var name = 'Salt Master CPU'

var idVar = name + ':{{.Group}}'

var message = ' {{ index .Tags "host" }}:{{ .Name }}:{{ index .Tags "type_instance" }} is {{.Level}} value: {{ index .Fields "value" }} @ {{.Time}} {{.Tags}}'

var idTag = 'alertID'

var levelTag = 'level'

var messageField = 'message'

var durationField = 'duration'

var outputDB = 'chronograf'

var outputRP = 'autogen'

var outputMeasurement = 'alerts'

var triggerType = 'threshold'

var crit = 70

var data = stream |from() .database(db) .retentionPolicy(rp) .measurement(measurement) .groupBy(groupBy) .where(whereFilter) |window() .period(period) .every(every) .align() |mean('value') .as('value')

var trigger = data |alert() .crit(lambda: "value" < crit) .stateChangesOnly() .message(message) .id(idVar) .idTag(idTag) .levelTag(levelTag) .messageField(messageField) .durationField(durationField) .slack() .channel('#embassyalerts')

trigger |influxDBOut() .create() .database(outputDB) .retentionPolicy(outputRP) .measurement(outputMeasurement) .tag('alertName', name) .tag('triggerType', triggerType)

trigger |httpOut('output')`

yakneens commented 7 years ago

Any signs of life in this product?

yakneens commented 7 years ago

Ping

sbengo commented 7 years ago

Hi @llevar ,you are chaining a |mean('value') on your from() node with an empty groupby clause.

If I'm not wrong, as it occurs on InfluxDB, you are aggregating data depending on your groupby clause. If it is empty, it will generate the field mean without keeping the tags, so it is doing the mean of all of your values without depending on your tags (filtered before with a where clause, obviously)

If you want to preserve the tag, I think you can use the groupby clause with specified tag or *

Remember that you can always use the log node and the httpoutput to see what are you doing on each node statement