Closed flavray closed 4 years ago
Example runs:
$ time kafka-consumer-manager --cluster-type <> --cluster-name <> offset_get <redacted> --use-admin-client
Cluster name: <>, consumer group: <redacted>
Topic Name: <redacted> Total Distance: 12067145
Partition ID: 0
High Watermark: 12067145
Low Watermark: 0
Current Offset: 0
Offset Distance: 12067145
Percentage Distance: 100.0%
Topic Name: <redacted> Total Distance: 1850
Partition ID: 0
High Watermark: 197266485
Low Watermark: 0
Current Offset: 197264635
Offset Distance: 1850
Percentage Distance: 0.0%
real 0m5.371s
user 0m4.232s
sys 0m0.200s
$ time kafka-consumer-manager --cluster-type <> --cluster-name <> offset_get <redacted>
Cluster name: <>, consumer group: <redacted>
Topic Name: <redacted> Total Distance: 421
Partition ID: 0
High Watermark: 197269431
Low Watermark: 0
Current Offset: 197269010
Offset Distance: 421
Percentage Distance: 0.0%
real 4m15.218s
user 3m59.092s
sys 0m11.048s
Ha. I had forgotten about OffsetWriter
. The build should be fixed on the next run :)
Reading the __consumer_offsets topic at every run is slow and can add unwanted load on a Kafka cluster. The AdminClient allows to get the list of consumer groups and the list of topics from a group id in constance time.
This is currently a feature flag to keep backwards compatibility, but will later be enabled by default.
--use-admin-client