karafka / waterdrop

Standalone Karafka library for producing Kafka messages
https://karafka.io
Other
258 stars 39 forks source link

Introduce `reload_on_transaction_fatal_error` to reload the librdkafka after transactional failures #503

Closed mensfeld closed 4 months ago

mensfeld commented 4 months ago

This PR introduces the ability of the producer to reload its underlying client when in transactional mode. This allows us to deal with cases when something goes wrong and a given instance should not be used.

Since it is used only from inside locked transactions, this does not have race conditions.

Fencing is excluded to prevent some weird race conditions.

close https://github.com/karafka/waterdrop/issues/501