search

spring-cloud / spring-cloud-stream-binder-kafka

Spring Cloud Stream binders for Apache Kafka and Kafka Streams
Apache License 2.0
331 stars 301 forks source link

Spring Cloud stream binder kafka DLQ Handling for Batching doesn't work #1225

Closed Sachin111us closed 1 year ago

Sachin111us commented 1 year ago

I am using spring-cloud-stream-binder-kafka version 3.2.6 in my application. I have enabled the batch-mode so that I should get the messages in batches. While handling DLQ scenario I observed that, In my batch if I have 3 messages and if my last message fail then entire batch goes to DLQ. Although I am throwing BatchListenerFailedException with the correct index of the message that has failed.

After debugging further I observed in Class org.springframework.cloud.stream.binder.kafka KafkaMessageChannelBinder method getErrorMessageHandler throws ClassCastException (Can not cast LinkedList to ConsumerRecord). Below is that line.

https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/blob/31ea10683422d38d8185c6b236b8954200562b77/spring-cloud-stream-binder-kafka/src/main/java/org/springframework/cloud/stream/binder/kafka/KafkaMessageChannelBinder.java#L1211

Method StaticMessageHeaderAccessor.getSourceData(message) returns list of ConsumerRecord but in the code it is expecting single consumer record. Is this a bug? If not can somebody help me to find how to resolve this issue.

Appreciate your early response.

Thanks

Sachin111us commented 1 year ago

Ignore this issue. It was a configuration issue from my side. I have batch-mode enabled and I have enabled enableDLQ as well. After disabling enableDLQ it worked

rajadilipkolli commented 1 year ago

Hi @Sachin111us , do you have working example?