KAFKA-17154: New consumer subscribe may join group without a call to consumer.poll

apache / kafka

Mirror of Apache Kafka

Apache License 2.0

28.39k stars 13.84k forks source link

KAFKA-17154: New consumer subscribe may join group without a call to consumer.poll #17165

Open FrankYang0529 opened 1 week ago

FrankYang0529 commented 1 week ago

To fulfill "rebalances will only occur during an active call to KafkaConsumer#poll(Duration)", we should not send JoinRequest after AsyncKafkaConsumer#subscribe. Add a flag subscriptionUpdated to AbstractMembershipManager#onSubscriptionUpdated. When calling AsyncKafkaConsumer#subscribe, set the flag to true. When calling AsyncKafkaConsumer#subscribe, send the JoinRequest if the flag is true.

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

TaiJuWu commented 6 days ago

Hi @FrankYang0529 , I opened a PR for this behavior. Could you take a look? Thanks a lot!

https://github.com/apache/kafka/pull/17175

FrankYang0529 commented 6 days ago

I opened a PR for this behavior. Could you take a look? Thanks a lot!

17175

Hi @TaiJuWu, thanks for the PR. I also have a similar test case testPollSendRequestForRebalance. We can also have your test after this PR is merged.

TaiJuWu commented 5 days ago

I opened a PR for this behavior. Could you take a look? Thanks a lot!

17175

Hi @TaiJuWu, thanks for the PR. I also have a similar test case testPollSendRequestForRebalance. We can also have your test after this PR is merged.

Hi @FrankYang0529 , I will close my PR because they do same thing and I leave a comment for testPollSendRequestForRebalance.

lianetm commented 2 days ago

Hey @FrankYang0529, so seems this draft is the PR we're keeping right? Let me know when it's ready for review and I'll be happy to take a look. Thanks!

FrankYang0529 commented 1 day ago

Hi @lianetm, yes, the PR is ready for review. Thank you.