influxdata / chronograf

Open source monitoring and visualization UI for the TICK stack
https://www.influxdata.com/time-series-platform/chronograf/
Other
1.51k stars 258 forks source link

':' in `idVar` is set by default #2681

Closed ono760 closed 6 years ago

ono760 commented 6 years ago

It looks like : in idVar is set by default when you use the Chronograf UI to create a Kapacitor alert. The variable idVar should not contain colons.

goller commented 6 years ago

Hey @ono760 , would you add the tickscript that was generated to this case?

ono760 commented 6 years ago

@goller

ID: chronograf-v1-8d8486e2-e280-4f7b-b668-7c43583b02dd
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 21 Dec 17 21:09 UTC
Modified: 21 Dec 17 23:57 UTC
LastEnabled: 21 Dec 17 23:57 UTC
Databases Retention Policies: ["infra-metrics"."autogen"]
TICKscript:
var db = 'infra-metrics'

var rp = 'autogen'

var measurement = 'system'

var groupBy = []

var whereFilter = lambda: TRUE

var name = 'foobar'

var idVar = name + ':{{.Group}}'

var message = 'yikes {{ index .Fields "value" }} {{.Time}}'

var idTag = 'alertID'

var levelTag = 'level'

var messageField = 'message'

var durationField = 'duration'

var outputDB = 'chronograf'

var outputRP = 'autogen'

var outputMeasurement = 'alerts'

var triggerType = 'threshold'

var crit = 0.5

var data = stream
|from()
.database(db)
.retentionPolicy(rp)
.measurement(measurement)
.groupBy(groupBy)
.where(whereFilter)
|eval(lambda: "load1")
.as('value')

var trigger = data
|alert()
.crit(lambda: "value" > crit)
.stateChangesOnly()
.message(message)
.id(idVar)
.idTag(idTag)
.levelTag(levelTag)
.messageField(messageField)
.durationField(durationField)
.sensu()

trigger
|influxDBOut()
.create()
.database(outputDB)
.retentionPolicy(outputRP)
.measurement(outputMeasurement)
.tag('alertName', name)
.tag('triggerType', triggerType)

trigger
|httpOut('output')

DOT:
digraph chronograf-v1-8d8486e2-e280-4f7b-b668-7c43583b02dd {
graph [throughput="16.00 points/s"];

stream0 [avg_exec_time_ns="0s" errors="0" working_cardinality="0" ];
stream0 -> from1 [processed="913794"];

from1 [avg_exec_time_ns="1.741304ms" errors="0" working_cardinality="0" ];
from1 -> eval2 [processed="913794"];

eval2 [avg_exec_time_ns="320.774µs" errors="456896" working_cardinality="1" ];
eval2 -> alert3 [processed="456898"];

alert3 [alerts_triggered="81448" avg_exec_time_ns="653.87µs" crits_triggered="40724" errors="0" infos_triggered="0" oks_triggered="40724" warns_triggered="0" working_cardinality="1" ];
alert3 -> http_out5 [processed="81448"];
alert3 -> influxdb_out4 [processed="81448"];

http_out5 [avg_exec_time_ns="658.706µs" errors="0" working_cardinality="1" ];

influxdb_out4 [avg_exec_time_ns="91.548µs" errors="0" points_written="81429" working_cardinality="0" write_errors="0" ];
}
goller commented 6 years ago

@ono760 we've had that idVar style for quite some time... what errors are you seeing?

ono760 commented 6 years ago

@goller

Please note that we tested this script out locally and found that it reports two errors in our system:

First if the sensu addr config contains http:// I get an error like this:

E! failed to send event to Sensu address http://localhost:3030: too many colons in address

Second if the .id() of the alert contains a colon : I get an error like this:

E! failed to send event to Sensu invalid name "m:nil" for sensu alert. Must match ^[\w\.-]+$

After removing the : in idVar, was able to get Kapacitor to send this:

{"handlers":null,"name":"loadanil","output":" system nil","source":"Kapacitor","status":2}

goller commented 6 years ago

@ono760 , ok I have a fix for the invalid name bug. Regarding the Sensu address one... how are you setting that?

ono760 commented 6 years ago

@goller

"addr": "10.56.4.205:3030"

goller commented 6 years ago

@ono760 I read kapacitor's sensu source code here: https://github.com/influxdata/kapacitor/blob/5d228b205dc4d5624b5b90208f320a1bbdc6e9c1/services/sensu/service.go#L104

It appears that the address must be "host:port" and not "scheme://host:port"

The docs don't call it out here:

https://docs.influxdata.com/kapacitor/v1.4//nodes/alert_node/#sensu ... but probably should.

@nathanielc , is there documentation that says that the sensu host should be host:port? /cc @stevebang

goller commented 6 years ago

@ono760 ok! fixed in #2684