search

KonstantinCodes / messenger-kafka

Simple Kafka transport for Symfony Messenger.
MIT License
84 stars 35 forks source link

Throw a TransportException when timeout occurs with kafka producer flush #24

Closed Kelthan closed 3 years ago

Kelthan commented 3 years ago

If Kafka is not available when dispatching a message, no error is thrown by the transport. As a result, we have no information about a message not being dispatch and we can't save it to retry the dispatch later if we need to.

KonstantinCodes commented 3 years ago

Thank You!

KonstantinCodes commented 3 years ago

https://github.com/KonstantinCodes/messenger-kafka/commit/635122a3ea08cf873317cfe284220aafe0882325

Actually, what do you think of this? There are lots of different possible errors. Some are retryable and others aren't.

Perhaps we can build a retry mechanism?

Kelthan commented 3 years ago

From the librdkafka documentation I saw that the flush method could return only a timeout error code : https://docs.confluent.io/4.1.4/clients/librdkafka/classRdKafka_1_1Producer.html#afa33f8d26eb49f8cc7988397bbb2db42 or am I wrong ?

I think retry mechanism is not a good idea here. If an exception occurred we should let people decide how to handle it, based on their project needs.