When a Rafka consumer receives an OffsetsCommitted event it will log an array with information about the commited offsets. This array contains information about all partition/offset pairs in which the Rafka consumer has been subscribed to, regardless of the amount of data that have been actually commited by librdkafka.
For example, consider a topic with 12 partitions; the OffsetsCommitted event of a single partition/offset pair (part: 9, offset: 1), looks like the following snippet:
This extraneous information makes debugging quite difficult; we actually interested in the updated partitions and not about the unaffected ones.
This commit introduces a new utility named dropInvalidOffsets which will be used to trim the "unset" offsets (aka those of OffsetInvalid type) from the logging output.
The logging ouput of the OffsetsCommitted event for the aforementioned example will be simplified as presented below:
When a Rafka consumer receives an
OffsetsCommitted
event it will log an array with information about the commited offsets. This array contains information about all partition/offset pairs in which the Rafka consumer has been subscribed to, regardless of the amount of data that have been actually commited by librdkafka.For example, consider a topic with 12 partitions; the
OffsetsCommitted
event of a single partition/offset pair (part: 9, offset: 1), looks like the following snippet:This extraneous information makes debugging quite difficult; we actually interested in the updated partitions and not about the unaffected ones.
This commit introduces a new utility named
dropInvalidOffsets
which will be used to trim the "unset" offsets (aka those ofOffsetInvalid
type) from the logging output.The logging ouput of the
OffsetsCommitted
event for the aforementioned example will be simplified as presented below: