Currently this is unimplemented by Java Kafka client defaults to enable.idempotence = true, causing failure of produce.
We will implement this on the server by keeping track of producer id and sequence number in the kafka in operator.
Note that idempotent producer functionality is not designed to survive broker failure, only to help protect against the case where a client experiences temporary network problems on it's connection to the server causing it to retry a send which was successful.
jlerche
commented
1 week ago
Currently this is unimplemented by Java Kafka client defaults to enable.idempotence = true, causing failure of produce.
We will implement this on the server by keeping track of producer id and sequence number in the kafka in operator.
Note that idempotent producer functionality is not designed to survive broker failure, only to help protect against the case where a client experiences temporary network problems on it's connection to the server causing it to retry a send which was successful.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging#KIP98ExactlyOnceDeliveryandTransactionalMessaging-IdempotentProducerGuarantees