search

OpenTSDB / opentsdb-rpc-kafka

A set of OpenTSDB plugins for consuming from Apache Kafka
Apache License 2.0
44 stars 25 forks source link

How do I use it #17

Open Blue-53Hz opened 4 years ago

Blue-53Hz commented 4 years ago

Hello, do you have any examples for reference? I want to use it to synchronize data to opentsdb

muffix commented 4 years ago

There are detailed instructions in the README.

The code here needs to be compiled and the resulting JAR put into OpenTSDB's plugin directory (as defined in tsd.core.plugin_path in the OpenTSDB config). Then, you will need to add the configuration items described here to the same (OpenTSDB's) config file. OpenTSDB will load the plugin on startup and spin up Kakfa consumers that consume the topic you configured.

The crucial part is the deserialiser class. Each message (i.e. the raw bytes) from the Kafka topic is passed to the deserialiser configured for that topic. A JSON deserialiser is provided out of the box, but you can also make your own by implementing the net.opentsdb.data.deserializers.Deserializer interface in case you have something other than JSON (Protobuf, for example) in your Kafka topic.

The deserialiser only needs to return a List of TypedIncomingData (which can be Metric, Aggregate, Histogram, or Metrics) for each message. The plugin takes care of the rest.