Open blindspotbounty opened 1 year ago
There is an approximate solution for Transactional API for Kafka gsoc.
Some ideas that were expressed in https://github.com/swift-server/swift-kafka-gsoc/issues/78#issuecomment-1642042408
There are the following ideas:
UPD: after review API changed to the following:
To use kafka transactions, it is required to create KafkaTransactionalProducerConfiguration:
KafkaTransactionalProducerConfiguration
let config = KafkaTransactionalProducerConfiguration(transactionalID: "1234")
Similar to KafkaProducer, it is possible to create KafkaTransactionalProducer:
KafkaProducer
KafkaTransactionalProducer
let transactionalProducer = try await KafkaTransactionalProducer.makeTransactionalProducerWithEvents(config: config)
To commit transactions, it is simply required to call withTransaction(...):
withTransaction(...)
try await transactionalProducer.withTransaction { transaction in // Produce new messages: let newMessage = KafkaProducerMessage( topic: "<some topic>", value: "<some value>" ) try transaction.send(newMessage) ... // commit offsets: let partitionlist = RDKafkaTopicPartitionList() partitionlist.setOffset(topic: self.uniqueTestTopic, partition: message.partition, offset: Int64(message.offset)) try await transaction.send(offsets: partitionlist, forConsumer: consumer) ... }
There is an approximate solution for Transactional API for Kafka gsoc.
Some ideas that were expressed in https://github.com/swift-server/swift-kafka-gsoc/issues/78#issuecomment-1642042408
There are the following ideas:
UPD: after review API changed to the following:
To use kafka transactions, it is required to create
KafkaTransactionalProducerConfiguration
:Similar to
KafkaProducer
, it is possible to createKafkaTransactionalProducer
:To commit transactions, it is simply required to call
withTransaction(...)
: