Telegraf - How does Sarama select which broker to connect to when a list is supplied

[clever-trevor]

Question

The Telegraf monitoring agent has an output plugin which posts metrics to Kafka, utilising the Sarama project. https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kafka

Within the Telegraf config, you specify a variable containing a string of one or more Brokers as follows brokers = ["localhost:9092","10.1.2.3:9092","1.2.3.4:9092"]

What I'm not clear on is how the selection is made on which broker to connect to, and the behaviour when that broker becomes unavailable.

Additional Info I took a quick look at the code and within the Telegraf kafka plugin, I believe this is the line which creates the Kafka connection producerFunc: sarama.NewSyncProducer,

(Forgive me if this is wrong, I'm not a Go programmer)

I initially raised this with InfluxData who support Telegraf (link below) in case that adds any additional info but they were unable to help as the Sarama code is not written by them https://community.influxdata.com/t/how-does-telegraf-kafka-output-determine-what-broker-to-use-when-multiple-are-speecified/17342

[clever-trevor]

Any update here?

[AnuragJ05]

It looks like sarama picks the first one from the list and ignores the rest, Is this a bug?

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[dnwe]

That’s not the case. Sarama will shuffle those bootstrap addresses on request. It will cycle through them to gain connectivity and then uses the advertised listeners for each broker when determining how to connect to those