search

IBM / sarama

Sarama is a Go library for Apache Kafka.
MIT License
11.57k stars 1.76k forks source link

Plan to support Incremenal Cooperative Rebalance Protocol? #1858

Open chickensoup opened 3 years ago

chickensoup commented 3 years ago

Kafka supports consumer incremental cooperative rebalance protocal starting from version 2.4.0, and I'm wondering is there any plan for Sarama client to support this protocol?

KIP-429

ghost commented 3 years ago

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the master branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

ilyakaznacheev commented 3 years ago

I just want to bump this issue from closing. Cooperative rebalance support would be very useful to work with different deployment kinds.

lizthegrey commented 2 years ago

Definitely would also help our issues at Honeycomb where we've had to stop and start all workers at once rather than rolling restart to update

past-urbanist commented 2 years ago

Now Apache Kafka 3.0 has used CooperativeStickyAssignor as default assignor instead of RangeAssignor. It seems Sarama has fallen far behind it. It is much more efficient in CooperativeStickyAssignor.

ilyakaznacheev commented 2 years ago

Still waiting for it!

lavoiesl commented 2 years ago

I believe this issue is referring to https://cwiki.apache.org/confluence/display/KAFKA/KIP-341%3A+Update+Sticky+Assignor%27s+User+Data+Protocol ? That feature was added https://github.com/Shopify/sarama/pull/1416, so I believe this can be closed?

gofish commented 2 years ago

Incremental rebalance is described by KIP-429.

dnwe commented 1 year ago

Thanks to @napallday we have a work-in-progress implementation of the incremental cooperate rebalance protocol. If you'd like to test out the changes they are available under PR https://github.com/IBM/sarama/pull/2608

dberardo-com commented 3 months ago

any ETA for this feature ?