Open JohnPreston opened 4 months ago
So far the only "workaround" I have found is to forcibly close and create a new set of clients each time, which would be preferable not to have to do.
EDIT: tinkering on this, given that at some point it seems the admin client was set to close, what happens to the existing requests that might be in the queue? Is that what the Local: Timeout
might be referring to?
Same happens in confluent-kafka-go when using AdminClient's ListConsumerGroupOffsets
method.
If cluster configuration changes (e.g. node restart) app crashes with:
Assertion failed: (eonce->refcnt > 0), function rd_kafka_enq_once_del_source_return, file rdkafka_queue.h, line 1052.
bad thing about go that CGO errors cannot be handled and whole app just crashes completely
Description
This might be related to the same problem as in #3663 but on a different API call. I am trying to simply scrape my Kafka cluster with the admin and consumer client. However, very often, do I get
python: rdkafka_queue.h:1052: rd_kafka_enq_once_del_source_return: Assertion
eonce->refcnt > 0' failed.`The source code of the application I am trying to build can be found here
The problem is, you can't quite catch this exception, and the process just completely gets killed.
How to reproduce
At first, this seemed to be very random, it'd happen after a while, or very quickly. As it turns out, tweaking the code to create an AdminClient each time before calling on the functions, instead of re-using and existing admin client, one leads to this problem every single time, right away.
Config
Checklist
IMPORTANT: We will close issues where the checklist has not been completed.
Please provide the following information:
<REPLACE with e.g., v0.10.5 or a git sha. NOT "latest" or "current">
<REPLACE with e.g., message.timeout.ms=123, auto.reset.offset=earliest, ..>
Logs
EDIT 1: This is repeatable as soon as the DescribeConsumerGroups times outs, everything goes wrong