Open AlexeyRaga opened 1 year ago
Hey @AlexeyRaga , thanks for reporting. I agree that if 10 partitions are assigned and you are looking at the topic lag we should either make clear that two partitions have no committed offsets at all, or we consider these partitions with a log of high watermark - low watermark.
What Console version were you using here? I'll try to reproduce this soon and add this case to our owl-shop demo application as well.
I am using redpandadata/console:v2.2.5
Related: #447
It looks like
console
reports total lag and partitions incorrectly.I have one consumer running in one thread consuming one topic with 10 partitions:
As you can see,
console
understands that all 10 partitions are assigned, but in the list below it reportsassigned partitions: 8
. The list also only contain 8 partitions and not 10.I believe that in the list it only shows partitions for which offsets are committed, completely ignoring partitions that haven't been consumed yet.
This screenshot of the
Members
tab supports this hypothesis:It shows all 10 partitions, but reporting
lag: 0
for the partitions that my consumer hasn't consumed from or committed offsets for.Therefore
Total lag
metric is unreliable and somehow semi-random.Expected behavior:
top watermark - consumer offset
for partitions that have offsets committed (does it currently)top watermark - bottom watermark
ortop watermark - 0
for the partitions that the consumer hasn't committed offsets forTotal lag
is calculated as a sum of the above.