bsm / sarama-cluster

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 [DEPRECATED]
MIT License
1.01k stars 222 forks source link

Closed connection to broker , how to reconnection #280

Closed 24 closed 6 years ago

24 commented 6 years ago

hi , i found sarama-cluster will close connection .

2018/11/23 11:28:20 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2018/11/23 11:28:20 Initializing new client
2018/11/23 11:28:20 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2018/11/23 11:28:20 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2018/11/23 11:28:20 client/metadata fetching metadata for all topics from broker biz1.kafka:9091
2018/11/23 11:28:20 Connected to broker at biz1.kafka:9091 (unregistered)
2018/11/23 11:28:20 client/brokers registered new broker #2 at 10.136.24.103:9091
2018/11/23 11:28:20 client/brokers registered new broker #1 at 10.136.24.206:9091
2018/11/23 11:28:20 Successfully initialized new client
2018/11/23 11:28:20 client/coordinator requesting coordinator for consumergroup censor from biz1.kafka:9091
2018/11/23 11:28:20 client/coordinator coordinator for consumergroup censor is #1 (10.136.24.206:9091)
2018/11/23 11:28:20 client/metadata fetching metadata for all topics from broker biz1.kafka:9091
2018/11/23 11:28:20 client/coordinator requesting coordinator for consumergroup censor from biz1.kafka:9091
2018/11/23 11:28:20 client/coordinator coordinator for consumergroup censor is #1 (10.136.24.206:9091)
2018/11/23 11:28:20 cluster/consumer  rebalance
2018/11/23 11:28:20 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2018/11/23 11:28:20 Connected to broker at 10.136.24.206:9091 (registered as #1)
2018/11/23 11:28:42 cluster/consumer sarama-aafce704-b5b7-4fd9-9de8-d35726337588 consume bizType-media/0 from 32
2018/11/23 11:28:42 cluster/consumer sarama-aafce704-b5b7-4fd9-9de8-d35726337588 consume bizType-media/1 from 49
2018/11/23 11:28:42 consumer/broker/1 added subscription to bizType-media/0
2018/11/23 11:28:42 consumer/broker/1 added subscription to bizType-media/1

2018/11/23 11:48:20 client/metadata fetching metadata for all topics from broker biz1.kafka:9091
2018/11/23 11:58:20 client/metadata fetching metadata for all topics from broker biz1.kafka:9091
......
2018/11/23 14:07:37 consumer/broker/1 closed dead subscription to bizType-media/0
2018/11/23 14:07:37 consumer/broker/1 closed dead subscription to bizType-media/1
2018/11/23 14:07:37 Closing Client
2018/11/23 14:07:37 Closed connection to broker 10.136.24.206:9091
2018/11/23 14:07:37 Closed connection to broker biz1.kafka:9091

i dno't know why closed .when i add config.ClientID="xxx" ,then will ok .mybe this not reason?

my config


    config := cluster.NewConfig()
    config.Consumer.Return.Errors = true
    config.Group.Return.Notifications = true
    config.Group.Offsets.Retry.Max = 100
    config.Metadata.Retry.Max = 20
    config.ClientID = "censor_media_consumer"
    //config.Metadata.Retry.Backoff = 2 * time.Second

how i know Closing Client Event, or Closed connection to broker ,if close , then i want restart client . or reconnection client .

because

    // consume errors
    go func() {
        for err := range consumer.Errors() {
            logger.KafkaConsumerLogger.Errorln("[kafka.consumer]Error: %s\n", err.Error())
        }
    }()

    // consume notifications
    go func() {
        for ntf := range consumer.Notifications() {
            logger.KafkaConsumerLogger.Infof("[kafka.consumer]Rebalanced: %+v\n", ntf)
        }
    }()

they are nothing notifications .only found sarama.Logger found close client or close connection

sorry , I don't know much about kafka. please guide me ,how to receive client close or broker close notifications? , then code reconnection or auto restart Consumer cluster.client ? . and how to print detail sarama-cluster log ?

thanks !

dim commented 6 years ago

@24 my apologies, but since https://github.com/Shopify/sarama/pull/1099 was merged and released, this library is officially deprecated.

24 commented 6 years ago

@dim thanks