getsentry / arroyo

A library to build streaming applications that consume from and produce to Kafka.
https://getsentry.github.io/arroyo/
Apache License 2.0
41 stars 7 forks source link

fix(dlq): Gracefully handle case of no valid messages #291

Closed nikhars closed 12 months ago

nikhars commented 12 months ago

In the case where all messages on the kafka pipeline are bad, it is possible that there is no valid message. Today, the code would crash because of division by zero error. Instead, we should reject the messages and let the consumers backlog because zero valid messages indicates systemic problems in the pipeline.