KafkaProducer: we want to expose a general KafkaProducerEvent type
in the AsyncSequence that was formerly just for message
acknowledgements. This enables us to add more events such as statistics
in the future.
The reason behind having a single AsyncSequence for all event types
instead of having separate AsyncSequences for each event type is that
we need to ensure that all events that we subscribe to in librdkafka
are actually consumed by the AsyncSequence in our KafkaProducer
type. Otherwise we could run out of memory.
Now by giving the user the entire events AsyncSequence, they decide if
they want to consume event or drop it.
Note: Logs will be consumed regardless of the
KafkaProducerEventsAsyncSequence
Motivation:
KafkaProducer
: we want to expose a generalKafkaProducerEvent
type in theAsyncSequence
that was formerly just for message acknowledgements. This enables us to add more events such as statistics in the future.The reason behind having a single
AsyncSequence
for all event types instead of having separateAsyncSequence
s for each event type is that we need to ensure that all events that we subscribe to inlibrdkafka
are actually consumed by theAsyncSequence
in ourKafkaProducer
type. Otherwise we could run out of memory.Now by giving the user the entire events
AsyncSequence
, they decide if they want to consume event or drop it.Modifications:
enum
KafkaProducerEvent
KafkaMessageAcknowledgements
->KafkaProducerEvents
KafkaProducer.makeProducerWithAcknowledgements
->.makeProducerWithEvents