li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
BSD 2-Clause "Simplified" License
131
stars
52
forks
source link
Bypass large message processing when passthrough is enabled for consumer #193
Problem: When BMM testing the passthrough feature, it turns out during ConsumerRecordsProcessor.handleConsumerRecord, we are transforming the underlying PassthroughConsumerRecord into ConsumerRecord type, thus causing some type cast issue in the caller.
Fix: When passthrough is enabled on consumer side, ideally we should bypass the large message processing part and just return the consumed record directly to the caller.
Test: added integration test to verify the return record type when enable.shallow.iterator is set to true for consumer.
Problem: When BMM testing the passthrough feature, it turns out during ConsumerRecordsProcessor.handleConsumerRecord, we are transforming the underlying PassthroughConsumerRecord into ConsumerRecord type, thus causing some type cast issue in the caller.
Fix: When passthrough is enabled on consumer side, ideally we should bypass the large message processing part and just return the consumed record directly to the caller.
Test: added integration test to verify the return record type when enable.shallow.iterator is set to true for consumer.