[Improvement] Add a way to check a consumer group's lag

[sarut0bi]

Hi,

It would be great to have a feature to monitor a reader's lag. Considering, once lag is 0 for a given consumer group, then All produced data have been consumed.

Actually, I've set a reader of specific consumer group in my k6 script and I start the trends measure once I've done my writer.produce then the trend measure is the read time of k6 script / the consumer group I use in my script, not the app consumer group I want to watch.

Hope this feature request is clear and/or even better if some solution already exists

[mostafa]

Hey @sarut0bi,

If you connect your k6 with an external metrics output, like Prometheus or InfluxDB, you'll be able to see in real-time what the reader's lag is. The emitted metrics I am referring to is named kafka_reader_lag and it is a gauge metric.

[strowi]

Just though about a similar "problem". I just made first steps with kafka and xk6-kafka, so far i was a really easy and nice way to check if my config was correct (esp. authentication vs. modifying a client.properties).

Now i started testing consumer-groups, i noticed a metric from the kafka-exporter which shows "Messages behind". I''m guessing this is different from the kafka_reader_lag and shows the message-lag when producer and consumer run in parallel.

But the example-scripts seem to be running the producer first and then the consumer.

[mostafa]

Hey,

I suppose your question is answered, I'll close this issue. If you have further questions related to the same issue, please re-open this, otherwise open a new one.