Lag and partitions offsets are presented incorrectly

[AlexeyRaga]

It looks like console reports total lag and partitions incorrectly.

I have one consumer running in one thread consuming one topic with 10 partitions:

As you can see, console understands that all 10 partitions are assigned, but in the list below it reports assigned partitions: 8. The list also only contain 8 partitions and not 10.

I believe that in the list it only shows partitions for which offsets are committed, completely ignoring partitions that haven't been consumed yet.

This screenshot of the Members tab supports this hypothesis:

It shows all 10 partitions, but reporting lag: 0 for the partitions that my consumer hasn't consumed from or committed offsets for.

Therefore Total lag metric is unreliable and somehow semi-random.

Expected behavior:

• Both lists having the correct number of partitions
• Lag is displayed as:
  • top watermark - consumer offset for partitions that have offsets committed (does it currently)
  • top watermark - bottom watermark or top watermark - 0 for the partitions that the consumer hasn't committed offsets for
  • Total lag is calculated as a sum of the above.

[weeco]

Hey @AlexeyRaga , thanks for reporting. I agree that if 10 partitions are assigned and you are looking at the topic lag we should either make clear that two partitions have no committed offsets at all, or we consider these partitions with a log of high watermark - low watermark.

What Console version were you using here? I'll try to reproduce this soon and add this case to our owl-shop demo application as well.

[AlexeyRaga]

I am using redpandadata/console:v2.2.5

[twmb]

Related: #447