jmxtrans / jmxtrans

jmxtrans
MIT License
1.71k stars 503 forks source link

kafka writer that uses InfluxDB line protocol? #547

Open jeroenjacobs79 opened 7 years ago

jeroenjacobs79 commented 7 years ago

Hi,

I currently uses Telegraf to gather system metrics, and I use Kafka as a transport layer. I have two dedicated instances (using Telegraf Kafka consumer, working as consumer group), that get the Telegraf metrics from the Kafka topic, and put them in the InFlux database. I prefer this approach compared to letting Telegraf report to Influx directly.

Is something like this possible with jmxtrans? I see jmxtrans has an influxdb output, but that's not exactly what I need. But since jmxtrans already has code for the influxdb line protocol, would it be difficult to put the metrics on a kafka topic?

gehel commented 7 years ago

First, keep in mind that I know mostly nothing about kafka, influxdb and telegraf, and even less about your setup and what exactly you are trying todo. Some guesswork in my answer...

We already have an influxdb writer and we also already have a kafka writer. The kafka wirter is probably not modular enough to be reused, but that's a nice refactoring exercise.

Feel free to send a pull request!