influxdata / kapacitor

Open source framework for processing, monitoring, and alerting on time series data
MIT License
2.31k stars 492 forks source link

Support for HTTP PUT, PATCH, DELETE and other methods #1534

Open rapport opened 7 years ago

rapport commented 7 years ago

We are sending a HTTP PUT to make Hue Lights go red when an alarm goes off. Is there a way to do this directly from Kapacitor? (I found and use the HTTP POST but am missing other HTTP methods). For example in JavaScript I would send a PUT like this to switch the lights on and make them red:

var xmlhttp = new XMLHttpRequest(); xmlhttp.open("PUT", 'http://myhuelights/api//lights//state'); xmlhttp.send('{"on":true, "sat":254, "bri":254,"hue":5448,"alert": "none"}');

It would also be useful to have all the methods for various other use cases. For example I would love to be able to simply recursively PATCH our alerts via the kapacitor API as they get fired as that is such an efficient way to change and improve many types of alerts dynamically.

leopoul commented 7 years ago

Hi,

In our case, we have several python scripts which perform specialized external API tasks. For example, whenever an alert condition is reached, we execute a series of scripts that interface with the APIs of other tools. We do this via exec() whereas our python scripts live under /usr/bin. To get the .id, .message, tags, timestamp, alert level, etc we parse the stdin as json. As an example, we want to include grafana screenshots in our alert email. For that we use a python script that grabs the related grafana graph rendered png and includes it in the alert email with alert details and the link to the grafana graph. In your case you could have something similar where your python script would invoke the requests lib to perform various calls depending on the alert level.

Best, Leo

nathanielc commented 7 years ago

@rapport I agree with @leopoul suggestion here. While it is simple to change the HTTP method used during the request, changing the JSON structure is not. As such using a python script or similar gives you complete control over how the alert request is made.

rapport commented 7 years ago

Many thanks for the workaround ideas which are really useful... I will implement something like this for now. Regards adding support for REST. As REST is the architecture basis of the web I think it is a special case and should be supported out of the box if at all possible. I expect other user would also put this as high priority compared with other alert nodes. I found it fairly easy to format email messages and wonder if it is possible to use the same parser for sending content. Supporting content types would also be required. I think it defintely makes sense for our type of application to support all HTTP method which is an IOT - for IOT managing the 'state' and 'security' of the system is difficult without having something build-in/out-of-the-box and less very secure using exec().

ghost commented 6 years ago

Hi @nathanielc we are facing an issue while patching to script. Could you please help on the below. https://github.com/influxdata/kapacitor/issues/1639

We are trying to patch this "script": "batch\n |query('''\n SELECT count(response_time) as count\n from "sdp_monitoring"."13m"."net_response"\n where "type" = 'kafka'\n ''')\n .groupBy('cluster', 'host')\n .period(45s)\n .every(10s)\n |where(lambda: "count" \u003e 0)\n |deadman(1.0, 30s)\n .stateChangesOnly()\n .exec('/kapacitor/tick-scripts/hpomi.py', 'Kafka Process Down Alert', 'Kafka Process Down', 'Kafka Process Down')\n\n// stream\n// |from()\n// .database('sdp_monitoring')\n// .retentionPolicy('13m')\n// .measurement('net_response')\n// .where(lambda: "type" == 'kafka')\n// .groupBy('cluster', 'host')\n// |deadman(1.0, 30s)\n// .stateChangesOnly()\n// .exec('/kapacitor/tick-scripts/hpomi.py', 'Kafka Process Down Alert', 'Kafka Process Down', 'Kafka Process Down')\n\n"

adding this to the above AND "cluster" != 'tsys' but we are facing issue even while patching with same thing mentioned above. Below is the error we escaped brackets and \n even though we are facing error with script. Could anyone please have a look on this.

curl --request PATCH -H "Accept: application/json" --data '{"script": "batch/ \n |query('''/ \n SELECT count(response_time) as count/ \n from "sdp_monitoring"."13m"."net_response"/ \n where "type" = 'kafka'/ \n ''')/ \n .groupBy('cluster', 'host')/ \n .period(45s)/ \n .every(10s)/ \n |where(lambda: "count" \u003e 0)/ \n |deadman(1.0, 30s)/ \n .stateChangesOnly()/ \n .exec('/kapacitor/tick-scripts/hpomi.py', 'Kafka Process Down Alert', 'Kafka Process Down', 'Kafka Process Down')/ \n/ \n// stream/ \n// |from()/ \n// .database('sdp_monitoring')/ \n// .retentionPolicy('13m')/ \n// .measurement('net_response')/ \n// .where(lambda: "type" == 'kafka')/ \n// .groupBy('cluster', 'host')/ \n// |deadman(1.0, 30s)/ \n// .stateChangesOnly()/ \n// .exec('/kapacitor/tick-scripts/hpomi.py', 'Kafka Process Down Alert', 'Kafka Process Down', 'Kafka Process Down')/ \n/ \n"}' http://localhost:9092/kapacitor/v1/tasks/kafkaproc_alert curl: (6) Could not resolve host: \n; Name or service not known curl: (6) Could not resolve host: SELECT; Name or service not known curl: (6) Could not resolve host: count(response_time); Name or service not known curl: (6) Could not resolve host: as; Name or service not known curl: (6) Could not resolve host: count; Name or service not known curl: (6) Could not resolve host: \n; Name or service not known curl: (6) Could not resolve host: from; Name or service not known curl: (6) Could not resolve host: "sdp_monitoring"."13m"."net_response"; Name or service not known curl: (6) Could not resolve host: \n; Name or service not known curl: (6) Could not resolve host: where; Name or service not known curl: (6) Could not resolve host: "type"; Name or service not known curl: (6) Could not resolve host: =; Name or service not known curl: (6) Could not resolve host: kafka; Name or service not known curl: (6) Could not resolve host: \n; Name or service not known curl: (6) Could not resolve host: ); Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (6) Could not resolve host: Down; Name or service not known curl: (6) Could not resolve host: Alert, Kafka; Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (6) Could not resolve host: Down, Kafka; Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (6) Could not resolve host: Down); Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (6) Could not resolve host: Down; Name or service not known curl: (6) Could not resolve host: Alert, Kafka; Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (6) Could not resolve host: Down, Kafka; Name or service not known curl: (6) Could not resolve host: Process; Name or service not known curl: (3) [globbing] unmatched close brace/bracket at pos 18 { "error": "invalid JSON" }

rapport commented 6 years ago

Invalid JSON sounds like you are not escaping some of the script correctly probably it is the double quotes unescaped in the wrong place:

For instance for "script": "batch\n |query('''\n SELECT count(response_time) as count\n from "sdp_monitoring"

check the double quotes inside the script value as for starter you are not opening and closing the value correctly with double quote as you have unescaped double quotes before and ending sdp_monitoring.