search

spring-cloud / spring-cloud-stream-binder-kafka

Spring Cloud Stream binders for Apache Kafka and Kafka Streams
Apache License 2.0
331 stars 301 forks source link

Producer initiated Kafka Transactional Producer doesn't include group,topic,partition in transactional.id #655

Closed Woodham closed 5 years ago

Woodham commented 5 years ago

I am experimenting with transactionIdPrefix, and it seems to correctly append the group id, topic name and partition to the Kafka transactional.id (for zombie fencing) when the transaction is initiated by a consumer, but it seems like it also creates a producer with transactional.id <prefix>0 which it uses for any messages sent that weren't initiated by a consumer, regardless of the topic, partition or group id.

Am I interpreting this correctly? And is this desired behaviour?

garyrussell commented 5 years ago

Github issues are for reporting bugs and asking for new features; questions should be asked on Stack Overflow, tagged with spring-kafka and/or spring-cloud-stream.

Yes, this is how the prefix is used. When producing a record outside of the context of a consumer thread, we have no context (the consumer group, topic, and partition from the inbound record).

Woodham commented 5 years ago

Apologies I wasn't 100% sure if this was a bug or not. Thanks for the clarification.

garyrussell commented 5 years ago

No problem; if you don't care about zombie fencing, you can set the producer factory producerPerConsumerPartition to false and they'll all get the second style id.