Yelp / kafka-utils

Apache License 2.0
313 stars 127 forks source link

Allow kafka-rolling-restart script to be executed in clusters with SSL our SASL enabled #253

Closed dinegri closed 4 years ago

dinegri commented 4 years ago

The goal of this PR is to be able to perform rolling restart in a Kafka Cluster using SSL or SASL

For plaintext broker when you get metadata from Zookeeper, the result is like that (this result is used to get broker's hostname):

{
   "jmx_port":-1,
   "timestamp":"1312312312",
   "endpoints":[],
   "host":"broker-hostname:9092",
   "version":2,
   "port":-1
}

for broker using SSL is like that:

{
   "listener_security_protocol_map":{
      "SSL":"SSL"
   },
   "endpoints":[
      "SSL://broker-hostname:9093"
   ],
   "jmx_port":-1,
   "host":null,
   "timestamp":"1584750310126",
   "port":-1,
   "version":4
}

for broker using SSL with internal and external mapping is like that:

{
   "listener_security_protocol_map":{
      "SSL":"SSL"
   },
   "endpoints":[
      "INTERNAL://broker-hostname:9093",
      "EXTERNAL://broker-hostname:9093"
   ],
   "jmx_port":-1,
   "host":null,
   "timestamp":"1584750310126",
   "port":-1,
   "version":4
}

for broker using SASL is like that:

{
   "listener_security_protocol_map":{
      "SSL":"SSL"
   },
   "endpoints":[
      "PLAINTEXTSASL://foo.mycorp.com:9092"
   ],
   "jmx_port":-1,
   "host":null,
   "timestamp":"1584750310126",
   "port":-1,
   "version":4
}

in order to be able to handle these result, this PR changes get_broker_metadata method to handle these outputs

Baisang commented 4 years ago

Sorry for the delay. It's been a crazy couple of weeks.

All of our tests passed internally so will merge.