search

linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.76k stars 801 forks source link

How to use SASL_PLAINTEXT (Kerberos) with Burrow #584

Open apellegr06 opened 5 years ago

apellegr06 commented 5 years ago

Is it possible to use SASL_PLAINTEXT (Kerberos) with Burrow and if yes how to configure the jaas file path ?

trijimonpr commented 4 years ago

Hi @apellegr06 - I have been trying to setup SASL_SSL with no luck far, but I think the configuration below should work for plaintext.

`[sasl.mysasl] username="jass_username" password="jaas_password"

[client-profile.test] client-id="burrow-test" kafka-version="0.10.2.1" sasl="mysasl"`

iMajna commented 4 years ago

There is no documentation about connecting to Kafka with keytabs or using sasl.mechanism GSSAPI. Although there is no documentation I can see following issue which is closed: #204 and reference here #283. This means that in Burrow 1.0 this communication should be possible.

@toddpalino is it possible to confirm this? Would be grateful :)

patelprakashp commented 4 years ago

@toddpalino - I would also like to know that do we support Kafka with keytabs or using sasl.mechanism GSSAP , If yes is there documentation or sample we can follow to specify keytab file. @iMajna @trijimonpr @apellegr06 - if you have any details please forward to me.

patelprakashp commented 4 years ago

@bai @timbertson - Do guys have any update here ? thanks..

chaoqun-wu commented 4 years ago

I am a Go noob,and I tried to use SASL_PLAINTEXT (Kerberos) with Burrow. I tried to use this patch：

diff --git a/core/internal/helpers/sarama.go b/core/internal/helpers/sarama.go
index d216b94..d45ec29 100644
--- a/core/internal/helpers/sarama.go
+++ b/core/internal/helpers/sarama.go
@@ -129,6 +129,25 @@ func GetSaramaConfigFromClientProfile(profileName string) *sarama.Config {
                saramaConfig.Net.SASL.Password = viper.GetString("sasl." + saslName + ".password")
        }

+       // Configure kerberos if enabled
+       if viper.IsSet(configRoot + ".kerberos") {
+               saslName := viper.GetString(configRoot + ".kerberos")
+               saramaConfig.Net.SASL.Enable = true
+
+               saramaConfig.Net.SASL.Mechanism = sarama.SASLTypeGSSAPI
+               saramaConfig.Net.SASL.GSSAPI.AuthType = sarama.KRB5_KEYTAB_AUTH
+               //sasl.kerberos.service.name
+               saramaConfig.Net.SASL.GSSAPI.ServiceName = viper.GetString("kerberos." + saslName + ".servicename")
+               //krb5.conf
+               saramaConfig.Net.SASL.GSSAPI.KerberosConfigPath = viper.GetString("kerberos." + saslName + ".krb5")
+               //realm
+               saramaConfig.Net.SASL.GSSAPI.Realm = viper.GetString("kerberos." + saslName + ".realm")
+               //keytab
+               saramaConfig.Net.SASL.GSSAPI.KeyTabPath = viper.GetString("kerberos." + saslName + ".keytab")
+               //username
+               saramaConfig.Net.SASL.GSSAPI.Username = viper.GetString("kerberos." + saslName + ".username")
+       }
+
        return saramaConfig
 }

and this is my Burrow config for Kerberos

[client-profile.myclient] #this client profile name is myclient
kafka-version="2.2.1"    #kafka server version
client-id="burrow-myclient" # a string to be passed to kafka as client Id
kerberos="myclient"

[kerberos.myclient]
servicename="kafka"
krb5="/etc/krb5.conf"
realm="SIT"
keytab="/root/kafka.keytab"
username="kafka"

I modified krb5.conf because of this 

##krb5.conf
default_tkt_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
default_tgs_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
permitted_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
udp_preference_limit = 1

Retrieving the appropriate tickets seems to go without any issues. However, when this function is executed I get some trouble.Here is which line throw EOF exception.

Logs：

{"level":"debug","ts":1587689029.6086605,"msg":"Error while performing GSSAPI Kerberos Authentication: EOF","name":"sarama"}

Maybe these can help.

https://github.com/Shopify/sarama/issues/1526

https://github.com/Shopify/sarama/issues/1658

alexanderdehes commented 4 years ago

I think the problem is that a raw GSSAPI request (GSS_API_INITIAL )is sent to the broker which is rejected because it is not encapsulated in a kafka protocol header. As the broker does not reply, you get an EOF. GSSAPI support for broker V1.0.0 and higher is not working because of this.

ghost commented 4 years ago

Have anyone found a workaround as yet?

tanuj83 commented 4 years ago

are we ready for SASL_SSL (kerberos)

chaoqun-wu commented 4 years ago

I am a Go noob,and I tried to use SASL_PLAINTEXT (Kerberos) with Burrow. I tried to use this patch：

diff --git a/core/internal/helpers/sarama.go b/core/internal/helpers/sarama.go
index d216b94..d45ec29 100644
--- a/core/internal/helpers/sarama.go
+++ b/core/internal/helpers/sarama.go
@@ -129,6 +129,25 @@ func GetSaramaConfigFromClientProfile(profileName string) *sarama.Config {
                saramaConfig.Net.SASL.Password = viper.GetString("sasl." + saslName + ".password")
        }

+       // Configure kerberos if enabled
+       if viper.IsSet(configRoot + ".kerberos") {
+               saslName := viper.GetString(configRoot + ".kerberos")
+               saramaConfig.Net.SASL.Enable = true
+
+               saramaConfig.Net.SASL.Mechanism = sarama.SASLTypeGSSAPI
+               saramaConfig.Net.SASL.GSSAPI.AuthType = sarama.KRB5_KEYTAB_AUTH
+               //sasl.kerberos.service.name
+               saramaConfig.Net.SASL.GSSAPI.ServiceName = viper.GetString("kerberos." + saslName + ".servicename")
+               //krb5.conf
+               saramaConfig.Net.SASL.GSSAPI.KerberosConfigPath = viper.GetString("kerberos." + saslName + ".krb5")
+               //realm
+               saramaConfig.Net.SASL.GSSAPI.Realm = viper.GetString("kerberos." + saslName + ".realm")
+               //keytab
+               saramaConfig.Net.SASL.GSSAPI.KeyTabPath = viper.GetString("kerberos." + saslName + ".keytab")
+               //username
+               saramaConfig.Net.SASL.GSSAPI.Username = viper.GetString("kerberos." + saslName + ".username")
+       }
+
        return saramaConfig
 }

and this is my Burrow config for Kerberos

[client-profile.myclient] #this client profile name is myclient
kafka-version="2.2.1"    #kafka server version
client-id="burrow-myclient" # a string to be passed to kafka as client Id
kerberos="myclient"

[kerberos.myclient]
servicename="kafka"
krb5="/etc/krb5.conf"
realm="SIT"
keytab="/root/kafka.keytab"
username="kafka"

I modified krb5.conf because of this

##krb5.conf
default_tkt_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
default_tgs_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
permitted_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
udp_preference_limit = 1

Retrieving the appropriate tickets seems to go without any issues. However, when this function is executed I get some trouble.Here is which line throw EOF exception.

Logs：

{"level":"debug","ts":1587689029.6086605,"msg":"Error while performing GSSAPI Kerberos Authentication: EOF","name":"sarama"}

Maybe these can help.

Shopify/sarama#1526

Shopify/sarama#1658

I am a Go noob,and I tried to use SASL_PLAINTEXT (Kerberos) with Burrow. I tried to use this patch：

diff --git a/core/internal/helpers/sarama.go b/core/internal/helpers/sarama.go
index d216b94..d45ec29 100644
--- a/core/internal/helpers/sarama.go
+++ b/core/internal/helpers/sarama.go
@@ -129,6 +129,25 @@ func GetSaramaConfigFromClientProfile(profileName string) *sarama.Config {
                saramaConfig.Net.SASL.Password = viper.GetString("sasl." + saslName + ".password")
        }

+       // Configure kerberos if enabled
+       if viper.IsSet(configRoot + ".kerberos") {
+               saslName := viper.GetString(configRoot + ".kerberos")
+               saramaConfig.Net.SASL.Enable = true
+
+               saramaConfig.Net.SASL.Mechanism = sarama.SASLTypeGSSAPI
+               saramaConfig.Net.SASL.GSSAPI.AuthType = sarama.KRB5_KEYTAB_AUTH
+               //sasl.kerberos.service.name
+               saramaConfig.Net.SASL.GSSAPI.ServiceName = viper.GetString("kerberos." + saslName + ".servicename")
+               //krb5.conf
+               saramaConfig.Net.SASL.GSSAPI.KerberosConfigPath = viper.GetString("kerberos." + saslName + ".krb5")
+               //realm
+               saramaConfig.Net.SASL.GSSAPI.Realm = viper.GetString("kerberos." + saslName + ".realm")
+               //keytab
+               saramaConfig.Net.SASL.GSSAPI.KeyTabPath = viper.GetString("kerberos." + saslName + ".keytab")
+               //username
+               saramaConfig.Net.SASL.GSSAPI.Username = viper.GetString("kerberos." + saslName + ".username")
+       }
+
        return saramaConfig
 }

and this is my Burrow config for Kerberos

[client-profile.myclient] #this client profile name is myclient
kafka-version="2.2.1"    #kafka server version
client-id="burrow-myclient" # a string to be passed to kafka as client Id
kerberos="myclient"

[kerberos.myclient]
servicename="kafka"
krb5="/etc/krb5.conf"
realm="SIT"
keytab="/root/kafka.keytab"
username="kafka"

I modified krb5.conf because of this

##krb5.conf
default_tkt_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
default_tgs_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
permitted_enctypes = aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96 des-cbc-md5 des-cbc-crc des3-cbc-sha1
udp_preference_limit = 1

Retrieving the appropriate tickets seems to go without any issues. However, when this function is executed I get some trouble.Here is which line throw EOF exception.

Logs：

{"level":"debug","ts":1587689029.6086605,"msg":"Error while performing GSSAPI Kerberos Authentication: EOF","name":"sarama"}

Maybe these can help.

Shopify/sarama#1526

Shopify/sarama#1658

Update sarama with a fix for #1697 , I can use SASL_PLAINTEXT (Kerberos) with Burrow now.