search

skroutz / rafka

Kafka proxy with a simple API, speaking the Redis protocol
https://engineering.skroutz.gr/blog/kafka-rails-integration/
GNU General Public License v3.0
8 stars 0 forks source link

Expose librdkafka stats to clients #28

Open agis opened 7 years ago

agis commented 7 years ago

The Go driver is able to report stats events that we could expose to clients (see https://github.com/confluentinc/confluent-kafka-go/commit/18b5a55f7ee663a3fc44a42ca937ed276de529d0). This will be useful for monitoring Rafka (eg. monitor lagging consumers).

To begin with, the server, we could implement 2 commands for consumer-specific monitoring:

  1. one to list all the active consumers
  2. one to fetch stats for a particular consumer

We need to decide which available Redis commands serve as good candidates for such a command. Possibly CLIENT LIST is good for (1).

agis commented 7 years ago

For now, what we want is a simple count of errors in the producers & consumers.