dockerzhang / incubator-inlong

Apache InLong - a one-stop data streaming platform
https://inlong.apache.org/
Apache License 2.0
0 stars 0 forks source link

[INLONG-149] Some of the consumers stop consuming their corresponding partitions and never release the partition to others #149

Closed dockerzhang closed 3 years ago

dockerzhang commented 3 years ago

We have experienced issues while we consuming a topic for a while, some of the consumers will wait forever and stop consuming from Tube.
The problem here is the corresponding partitions are being held and it will not trigger any repartition, therefore messages would be delayed in these partitions while the others are OK.

 

We have faced this issue in both 3.5.x and 3.8.x version recently, however we have used Tube for months and never have this issue before.
Another point here is we have 100+ topics and many consumers, only a few consumer groups are having this issue.

It is hard to reproduce since it just happened randomly.

 

JIRA link - [INLONG-149] created by marklaw