The message parser currently assumes that messages always have the magic byte set to 0. However, the Java producer of Kafka 0.7.0 generates messages with magic == 1.
Currently, the parser doesn't even check the value of the magic byte, and just assumes the version 0 format. Thus the message is parsed incorrectly (the last byte of the CRC32 is added as the first byte of the payload, and the CRC32 doesn't match).
This patch adds basic support for version 1. It doesn't yet interpret the value of the attributes byte (that would be a useful addition for the future).
acrosa
commented
12 years ago
The message parser currently assumes that messages always have the magic byte set to 0. However, the Java producer of Kafka 0.7.0 generates messages with magic == 1.
Currently, the parser doesn't even check the value of the magic byte, and just assumes the version 0 format. Thus the message is parsed incorrectly (the last byte of the CRC32 is added as the first byte of the payload, and the CRC32 doesn't match).
This patch adds basic support for version 1. It doesn't yet interpret the value of the attributes byte (that would be a useful addition for the future).