feat: Increase backpressure threshold to 5 seconds

getsentry / arroyo

A library to build streaming applications that consume from and produce to Kafka.

https://getsentry.github.io/arroyo/

Apache License 2.0

39 stars 6 forks source link

feat: Increase backpressure threshold to 5 seconds #340

Closed lynnagara closed 4 months ago

lynnagara commented 4 months ago

During inc-626 we saw excessive backpressure which caused the consumer to be paused and resumed a lot. This led to excessive network traffic as the local queue was purged and messages were re-downloaded. Increase the threshold for pausing to reduce the impact of this.