Multiple Kafka sink with Kerberos authentication failed.

[ffcactus]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

We use Vector to send message to 2 Kafka broker. These 2 brokers use Kerberos for authentication. No matter how we set the SASL or krb5.conf, there will be a kind of error message like this:

Server krbtgt/AAA@BBB not found in Kerberos database

In the error message, AAA and BBB come from different realm, for example, STGKAFKA100.REALM1.COM@KAFKA200.REALM2.COM

I was told that some Java library doesn't support multiple Kerberos realm. I'm not sure if Vector has same issue.

Configuration

Here is our krb5.conf

[logging]
   kdc = FILE:/etc/vector/krb5/krb5kdc.log
   admin_server = FILE:/etc/vector/krb5/kadmin.log
   default = FILE:/etc/vectorkrb5/krb5libs.log

[libdefaults]
   dns_lookup_realm = true
   dns_lookup_kdc = true
   ticket_lifetime = 24h
   forwardable = false
   allow_weak_crypto = true
   renew_lifetime = 7d
   kdc_timeout = 3000
   max_retries = 2
   clockskew = 120

[appdefaults]
   pam = {
      debug = false
      ticket_lifetime = 36000
      renew_lifetime = 36000
      forwardable = false
      krb4_convert = false
   }

[realms]
   KAFKA100.REALM.COM = {
      kdc = stg1-kdc.local
   }

   KAFKA200.REALM.COM = {
      kdc = stg2-kdc.com
   }

[domain_realm]
   .broker1.com = KAFKA100.REALM.COM
   .broker2.com = KAFKA200.REALM.COM

Version

v0.32.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

[ffcactus]

I think this issue is because Vector is acted as a single Kerberos client. If it needs to send message to different Kafka broker that belongs to different Kerberos realm, those two realm must be configured to use across realm authentication. However, sometimes it's impossible because physical network separation, or they belongs to different organizations.

Are we able to make Vector acts as multiple Kerberos clients when we have multiple Kafka sinks?