getsentry / arroyo

A library to build streaming applications that consume from and produce to Kafka.
https://getsentry.github.io/arroyo/
Apache License 2.0
39 stars 6 forks source link

ref: Add more metrics for slow rebalancing #310

Closed untitaker closed 7 months ago

untitaker commented 7 months ago

the unitless metrics are good because they are low overhead to measure, but in the case of rebalancing, the overhead does not matter. emit a metric value everytime partitions are assigned/revoked (and how often a callback decides to shutdown/create strategies as a result), and also emit a counter for how many partitions are being assigned