search

linkedin / li-apache-kafka-clients

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
BSD 2-Clause "Simplified" License
131 stars 52 forks source link

Kafka headers? #115

Closed caoimhindenais closed 5 years ago

caoimhindenais commented 5 years ago

Thanks for publishing the project on github. This is a question more than an issue.

What is the advantage of including meta information on the payload as part of the payload itself? Would it more naturally belong in Kafka headers? It could also help development/operarations identify large message easily (without having to deserialise the payload) …

 public LargeMessageSegment deserialize(String s, byte[] bytes) {
    int headerLength = 1 + LargeMessageSegment.SEGMENT_INFO_OVERHEAD +
radai-rosenblatt commented 5 years ago

this code predates headers in kafka. for now its still around for backwards compatibility