search

databricks / iceberg-kafka-connect

Apache License 2.0
220 stars 49 forks source link

Add Kerberos authentication option #236

Closed Dawnpool closed 7 months ago

Dawnpool commented 7 months ago

Hello, I was going to use this connector for hdfs, but I noticed it does not support Kerberos authentication. To address this, I added Kerberos authentication functionality along with other related options. In implementing this feature, I have used the existing hdfs sink connector codes as a reference, which already does support Kerberos authentication. (document link)

The example config would look like

iceberg.hdfs.authentication.kerberos: true,
iceberg.connect.hdfs.principal: "user@EXAMPLE.COM",
iceberg.connect.hdfs.keytab: "/tmp/user.keytab",
kerberos.ticket.renew.period.ms: 3600000

I understand there should be further tasks such as updating README and adding some test codes, but I wanted to share the core idea with you first for initial feedback. Please take a look. Thank you.

tabmatfournier commented 7 months ago

:wave: This repository is in the process of being moved to Iceberg core. Half of it has been moved already. I would suggest making a PR there as these files are already present in Iceberg core https://github.com/apache/iceberg/tree/e6a1a45624b8bd8246c6ef5601cae046015f4532/kafka-connect

Dawnpool commented 7 months ago

@tabmatfournier I see. I will make the same PR there. Thank you!