linkedin / li-apache-kafka-clients

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
BSD 2-Clause "Simplified" License
131 stars 52 forks source link

Adding original message size to the large message header value #172

Closed navina closed 4 years ago

navina commented 4 years ago

LargeMessageHeaderValue is missing messageSizeInBytes, which indicates the size of the original message. This is useful to validate the segments during assembling on the consumer side.
With the existing code, we will see failures when we try to assemble based on the metadata from the large message header. Not sure why this left out in the original patch that introduced the header value. Adding it now to ensure that segment validation can be done.