Open monimail opened 6 years ago
For monitoring kafka, I used logstash (with jmx plugin)
input {
jmx {
path => "/usr/share/logstash/resources/jmx"
polling_frequency => 60
type => "kafka-jmx"
nb_thread => 4
}
In the jmx directory I have the folowing json with all beans to be monitored
.... cat kafka.json
{
"host" : "kafka",
"port" : 9999,
"alias" : "ns",
"queries" : [
{
"object_name" : "kafka.server:type=*,name=*"
},
{
"object_name" : "kafka.server:type=*,name=*,topic=*"
},
{
"object_name" : "kafka.cluster:name=*,partition=*,topic=*,type=*"
},
{
"object_name" : "kafka.log:name=*,partition=*,topic=*,type=*"
} ]
}
Then you can insert that date in whatever database you like (and logstash support).
Hope this would be useful
Regards
Hello,
I am trying to monitor how Kafka behaves in terms of performance. I tried it with jolokia in the openbmp_aio container as described here : https://github.com/paksu/kafka-jolokia-telegraf-collector but jolokia never sends anything.
As a result, I installed kafka on my host server in order to try to install jolokia more easily, but now I'm getting those log lines :
I am also having issues with the openbmp_kafka container that was not accepting TCP connections from the other containers (as described in https://github.com/SNAS/website/issues/85).
Any help would be appreciated.
Thanks in advance.