I'm using the consumer group API and this method to commit offsets:
// CommitOffsets commits the provided topic+partition+offset combos to the
// consumer group coordinator. This can be used to reset the consumer to
// explicit offsets.
func (g *Generation) CommitOffsets(offsets map[string]map[int]int64) error {
Now I've been in the case where my Kafka brokers restarted and my code keeped trying calling CommitOffsets in a retry loop that I coded. The problem is that even when the brokers came back, CommitOffsets still failed with this error:
write tcp 10.68.48.18:32132->10.60.64.54:30093: use of closed network connection
It seems the connection is never restored in consumer group in case of problem. Anyway I suppose that the generation cannot be used anymore but my retry loop only stops on a "fatal" error like "unknown member id" and here I get a network error that seems to be "retryable".
Do you have suggestions on how to manage this kind of case?
I'm using the consumer group API and this method to commit offsets:
Now I've been in the case where my Kafka brokers restarted and my code keeped trying calling CommitOffsets in a retry loop that I coded. The problem is that even when the brokers came back, CommitOffsets still failed with this error:
It seems the connection is never restored in consumer group in case of problem. Anyway I suppose that the generation cannot be used anymore but my retry loop only stops on a "fatal" error like "unknown member id" and here I get a network error that seems to be "retryable".
Do you have suggestions on how to manage this kind of case?