"AND" operator with tick script templating and JSON

[badatya14]

Hi, I was trying to implement kapacitor alerting using templating. The challenge I am facing is using AND operator while passing var values in JSON file. For example, I cant find an option to add other value items in value field using AND. "where_filter": {"type": "lambda", "value": "\"cpu\" == 'cpu-total'"}, I want to see something like this: "where_filter": {"type": "lambda", "value": "\"cpu\" == 'cpu-total'" "and" "app": "\"app\" == 'demo'"}, Note - I tried, though the JSON validates fine but kapacitor doesn't like "and".

Appreciate any pointer on this.

[kprd]

Same here. or something like {"$and":[{"age":5},{"name":"Joe"}]}?

[desa]

@badatya14 can you include the template tickscript and the vars file so I can try to reproduce the issue. I cant quite tell if this is user error or an issue with Kapacitor itself.

[badatya14]

Template.tick var database = 'test' --> Not able to define this as string and pass value via json var measurement string --> Not able to define this as string and pass value via json var where_filter = lambda: TRUE var groups = [*] var field string var warn lambda var crit lambda var window = 10s var topic_name = '/tmp/alert_via_templating.log' var retentionpolicy = 'autogen' --> Not able to define this as string and pass value via json var levelTag = 'level' --> Not able to define this as string and pass value via json var durationField = 'duration' --> Not able to define this as string and pass value via json var idTag = 'alertID' --> Not able to define this as string and pass value via json

stream |from() .database(database) .retentionPolicy(retentionpolicy) .measurement(measurement) |default() .tag('app','') .tag('host','') .tag('region','') |where(where_filter) |groupBy(groups) |window() .period(window) .every(window) |mean(field) --> Not able to define this as string and pass value via json .as('stat') --> If this entry is removed, it works

|alert() .warn(warn) .crit(crit) .log(topic_name)

var.json { "measurement":{ "type":"string", "value":"cpu" }, "where_filter":{ "type":"lambda", "value":"\"cpu\" == 'cpu-total'"

---> Here the entry I would like to add is:

  value: "cpu" == 'cpu-total'
    AND "app" == 'abc'
    AND "region" == def'
    AND "zone" = 'test'
    ###

}, "groups":{ "type":"list", "value":[ { "type":"string", "value":"host" }, { "type":"string", "value":"dc" }, { "type":"string", "value":"app" }, { "type":"string", "value":"region" }, { "type":"string", "value":"env" } ] }, "field":{ "type":"string", "value":"usage_idle" }, "warn":{ "type":"lambda", "value":"\"mean\" < 20.0" }, "crit":{ "type":"lambda", "value":"\"mean\" < 15.0" }, "window":{ "type":"duration", "value":"10s" }, "topic_name":{ "type":"string", "value":"/tmp/template.log" } }

Hope this info helps you in reproducing the issue. Moreover how to define the .id() and .message() in var.json and refer to the template.tick. The points I mentioned above tick file, while defining the template it finds the value of the variable in the tick script itself for example database, retention policy, levelTag, durationField, idTag etc. And the important thing is I am not able to add AND operatioon in where_filter. Thanks in advance for your help.

[sbengo]

Hi @desa, I have been able to repro the issue, it seems to be related when loading JSON on template using -var function:

This is my TICKScript:

...
var whereFilter = lambda: ("instance" == 'fooInstance' AND "device" == 'fooDevice')
...
//TICKSCRIPT:
//================
var data = stream
    |from()
        .database(db)
        .retentionPolicy(rp)
        .measurement(measurement)
        .groupBy(groupBy)
        .where(whereFilter)
    |window()
        .period(period)
        .every(every)
        .align()
    |log()

With this TICK, I defined the following tasks:

Task defined from	Definition of var "whereFilter"	Result
Single-Task	var whereFilter = lambda: ("instance" == 'fooInstance' AND "device" == 'fooDevice')	OK
Templated-Task without VAR (same as Single-Task)	var whereFilter = lambda: ("instance" == 'fooInstance' AND "device" == 'fooDevice')	OK
Templated-Task with VAR from JSON	"whereFilter" : {"type" : "lambda", "value": "(\"instance\" == 'fooInstance' AND \"device\" == 'fooDevice')"}	NOOK: E! error while evaluating WHERE expression: left reference value "device" is missing value

It is quite strange behaviour because running show commands, it seems to load the exactly same params and it seems to be recognised by Kapacitor:

Task defined from	Result show command
Templated-Task without VAR (same as Single-Task)	whereFilter lambda ("instance" == 'fooInstance' AND "device" == 'fooDevice')
Templated-Task with VAR from JSON	whereFilter lambda ("instance" == 'fooInstance' AND "device" == 'fooDevice')

Thanks, Regards

[sbengo]

@nathanielc , @desa , are we doing something wrong?

Thank you, Regards

[bbczeuz]

Any workarounds or fixes here? Issue seems still present even with simpler setups: Log:

msg="failed to evaluate WHERE expression" service=kapacitor task_master=main task=pdu_inlet_others node=influxdb_out6 err="left reference value \"model\" is missing value"

('model' is an InfluxDB tag (pre 2.0) and it could be Null or some string (issue happens in both cases)) There is an influxdb_out node later, but the issue happens at the from()

# kapacitor show pdu_inlet_updu 
[...]
from1 [avg_exec_time_ns="0s" errors="224" working_cardinality="0" ];
from1 -> alert2 [processed="32"];
[...]

I use a templated task with VAR from YAML:

[...]
  where_filter:
    type: lambda
    value: "\"model\" =~ /^updu/"
[...]

Template:

[...]
var where_filter = lambda: TRUE
var groups = [*]
var data_raw = stream
    |from()
        .measurement(measurement)
        .where(where_filter)
        .groupBy(groups)
[...]