search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.13k stars 14.31k forks source link

Kerberos : switch MIT to Heimdal #26438

Open Jeoffreybauvin opened 2 years ago

Jeoffreybauvin commented 2 years ago

Apache Airflow version

2.3.4

What happened

Hi,

In my infrastructure, we are using Kerberos Heimdal for our keytabs. I can see that airflow seems to be using MIT...

I managed to switch to Heimdal by customizing the image : replacing krb5 packages with heimdal-clients.

ADDITIONAL_RUNTIME_APT_DEPS=""
RUNTIME_APT_DEPS="apt-transport-https apt-utils ca-certificates curl dumb-init freetds-bin gosu heimdal-clients ldap-utils libffi7 libldap-2.4-2 libsasl2-2 libsasl2-modules libssl1.1 locales lsb-release netcat openssh-client python3-selinux rsync sasl2-bin sqlite3 sudo unixodbc"

But the kinit command used by Airflow seems to be specific with MIT : 

airflow@7e253512ada7:/opt/airflow$ kinit -f -a -r 5m -k -t /opt/airflow/security/airflow.service.keytab -c /tmp/airflow_krb5_ccache airflow/dtl-airflowdocker01-d01.dc05@DC05 
5m@DC05's Password: 
kinit: krb5_get_init_creds: Client (5m@DC05) unknown

the "-r 5m" seems to be the problem. Adding --renewable or deleting -r 5m seems to fix this issue. But how to "fix" this properly ?

Regards,

What you think should happen instead

Either :

How to reproduce

Use kerberos with Heimdal.

Operating System

Debian (Docker)

Versions of Apache Airflow Providers

airflow-exporter==1.5.3
apache-airflow==2.3.4
apache-airflow-providers-amazon==5.0.0
apache-airflow-providers-apache-cassandra==3.0.0
apache-airflow-providers-apache-hdfs==3.1.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.3.0
apache-airflow-providers-common-sql==1.1.0
apache-airflow-providers-docker==3.1.0
apache-airflow-providers-elasticsearch==4.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-google==8.3.0
apache-airflow-providers-grpc==3.0.0
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-azure==4.2.0
apache-airflow-providers-mysql==3.2.0
apache-airflow-providers-odbc==3.1.1
apache-airflow-providers-postgres==5.2.0
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-sendgrid==3.0.0
apache-airflow-providers-sftp==4.0.0
apache-airflow-providers-slack==5.1.0
apache-airflow-providers-sqlite==3.2.0
apache-airflow-providers-ssh==3.1.0
google-cloud-orchestration-airflow==1.4.1

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Code of Conduct

potiuk commented 2 years ago

I changed the issue type from bug to "feature" - because that is what it is.

Why don't you implement this kind of switch @Jeoffreybauvin ? It does not seem to be a difficult one to implement, requires little Python knowledge and you could test it with your Heimdal Kerberos, so you seem to be a very good candidate to implement, test and contribiute - you could join the > 2100 contributors to Airflow - mostly users like you who had some problem and solved it by contributing.

I also marked it as "good-first-issue", so maybe someone will pick it but, the "surest" way to get it in is if you implement it (just wanted you to know so that you know that such features only get implemented if someone gets interested in implementing it, and you seem to be rather interested in implementing it).

Jeoffreybauvin commented 2 years ago

I can try, but I'm not sure what I need to do. Change the possible options for kinit ? Or switch to heimdal ?

potiuk commented 2 years ago

Add options only. Removing support for MIT kerberos would make a breaking change (and there are probably plenty of users using MIT one, so that wouldn't be empathetic towards them.

maheshsv commented 2 years ago

can you please assign it to me ,

potiuk commented 2 years ago

Assigned @maheshsv !