search

superstreamlabs / memphis

Memphis.dev is a highly scalable and effortless data streaming platform
https://docs.memphis.dev
Other
3.25k stars 216 forks source link

Feature: Producer Throttling #1365

Open keyboardcrimp opened 1 year ago

keyboardcrimp commented 1 year ago

Description

It would be nice to have some way to set a limit on the producers such that if there are X number of messages in a station then it will back off until they have been consumed. This will prevent build-up in the stations for large processing jobs where the bottleneck is the consumer.

Additional context

No response

Code of Conduct

yanivbh1 commented 1 year ago

Thank you @keyboardcrimp , It will be added to the backlog.