Fix to remove fetch queue messages that blocked the destroy of rdkafka instances

confluentinc / librdkafka

The Apache Kafka C/C++ library

Other

7.36k stars 3.11k forks source link

Fix to remove fetch queue messages that blocked the destroy of rdkafka instances #4724

Open emasab opened 1 month ago

emasab commented 1 month ago

Circular dependencies from a partition fetch queue message to the same partition blocked the destroy of an instance, that happened in case the partition was removed from the cluster while it was being consumed. Solved by purging internal partition queue, after being stopped and removed, to allow reference count to reach zero and trigger a destroy

emasab commented 1 month ago

How to reproduce: happening sporadically with test 0113 subtest n_wildcard. Run it it with TEST_DEBUG=all and until-fail.sh to see the refcnt not reaching zero.