logstash doesn't renew kafka ip when kafka pod restarts

[eblindeman]

When while logstash is running one of the kafka pods restarts (and gets a new IP) logstash doesn't renew de ip's which results in a consuming stop on the partition which resides on that pod.

for example: pod 2 was restarted the following warn is shown: [2019-10-15T13:13:37,101][WARN ][org.apache.kafka.clients.NetworkClient] [Consumer clientId=logstash-0, groupId=logstash-reporting] Connection to node 2 (kafka-2.broker.cts-dev.svc.cluster.local/xxx.xx.103.144:9092) could not be established. Broker may not be available.

the ip xxx.xx.103.144 was the old ip the new ip is xxx.xx.102.179

After logstash loses a broker it should use the bootstrap to resync the ips

• Version: Logstash 7.1.1 (own image only adds plugin) kafka broker 2.1.0
• Operating System: kubernetes 1.12.9
• Config File (if you have sensitive info, please remove it): logstash: appVersion: "7.1.1" image: repository: "registry.cts.appx.cloud/jenkins/cts-logstash-image" tag: "20190619081148" elasticsearch: host: elasticsearch-client port: 9200

[eblindeman]

When i upgraded tot logstash 7.4 with the new kafka input plugin the problem looks to be resolved. I will close this issue