airflow-helm / charts

The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
https://github.com/airflow-helm/charts/tree/main/charts/airflow
Apache License 2.0
647 stars 475 forks source link

cannot setup ldap with the error of 'Can't contact LDAP server' #779

Open zeddit opened 1 year ago

zeddit commented 1 year ago

Checks

Chart Version

8.8.0

Kubernetes Version

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.10+rke2r1", GitCommit:"e770bdbb87cccdc2daa790ecd69f40cf4df3cc9d", GitTreeState:"clean", BuildDate:"2023-05-18T15:22:04Z", GoVersion:"go1.19.9 X:boringcrypto", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.10+rke2r1", GitCommit:"e770bdbb87cccdc2daa790ecd69f40cf4df3cc9d", GitTreeState:"clean", BuildDate:"2023-05-18T15:22:04Z", GoVersion:"go1.19.9 X:boringcrypto", Compiler:"gc", Platform:"linux/amd64"}

Helm Version

version.BuildInfo{Version:"v3.12.1", GitCommit:"f32a527a060157990e2aa86bf45010dfb3cc8b8d", GitTreeState:"clean", GoVersion:"go1.20.4"}

Description

I was trying to setup ldap but failed. The web UI refresh to the login page with empty user and password text after I click sign in.

I have tried both method to integrate LDAP followed the instruction here https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-ldap

The first one is configured with STARTTLS:

AUTH_TYPE = AUTH_LDAP
AUTH_LDAP_SERVER = "ldap://ldap.xxx.tech"
AUTH_LDAP_USE_TLS = True

and the logs from airflow-web and ldap server are shown below:

# airflow web
airflow-web {manager.py:1110} ERROR - LDAP Could not activate TLS on established connection with ldap://ldap.xxx.tech
# ldap
64ef19a6 conn=32342 fd=12 ACCEPT from IP=10.0.18.32:53356 (IP=0.0.0.0:389)
64ef19a6 conn=32342 op=0 EXT oid=1.3.6.1.4.1.1466.20037
64ef19a6 conn=32342 op=0 STARTTLS
64ef19a6 conn=32342 op=0 RESULT oid= err=0 text=
64ef19a6 conn=32342 fd=12 TLS established tls_ssf=256 ssf=256
64ef19a6 conn=32342 fd=12 closed (connection lost)

it seems the ldap server accepted the connection but closed immediately, and airflow-web thought that it haven't connect with ldap server.

The second method is configured with LDAP over TLS (ldaps):

AUTH_TYPE = AUTH_LDAP
AUTH_LDAP_SERVER = "ldaps://ldap.xxx.tech"
AUTH_LDAP_USE_TLS = False

and logs from ldap and airflow-web are listed below:

# airflow web
airflow-web [2023-08-30T10:39:12.491+0000] {manager.py:1236} ERROR - {'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'}

# ldap log
64ef1c50 conn=32368 fd=12 ACCEPT from IP=10.0.18.33:53317 (IP=0.0.0.0:636)
64ef1c50 conn=32368 fd=12 TLS established tls_ssf=256 ssf=256
64ef1c50 conn=32368 fd=12 closed (connection lost)

similarly, airflow connect with ldap but do no search query, and user could not login to the airflow.

ldap server is correct, because other system like grafana works fine.

how could the bug comes from, and how to fix it. great thanks.

Relevant Logs

No response

Custom Helm Values

No response

zeddit commented 1 year ago

I have copy the code in https://github.com/dpgaspar/Flask-AppBuilder/blob/74f37e21a3c9c7ca7fb3e56f73759e3eaa2ead6b/flask_appbuilder/security/manager.py#L938 , and I found the auth has no wrong. besides, I get into the pod of airflow webserver and test the network connectivity with ldap server, there is no problem.

when using startls auth method, the code failed at

  if self.auth_ldap_use_tls:
      try:
          con.start_tls_s()
      except Exception:
          log.error(LOGMSG_ERR_SEC_AUTH_LDAP_TLS, self.auth_ldap_server)
          return None

however, when I executed the same code in other pods or on the other host directly, it works with no error.

zeddit commented 1 year ago

the problem has no progress yet.

thesuperzapper commented 1 year ago

@zeddit have you raised an issue with https://github.com/dpgaspar/Flask-AppBuilder?

zeddit commented 1 year ago

@thesuperzapper not yet. I think your advice is right, the root cause is at Flask-AppBuilder, I will raise an issue there. great thanks for your help.

stale[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had activity in 60 days. It will be closed in 7 days if no further activity occurs.

Thank you for your contributions.


Issues never become stale if any of the following is true:

  1. they are added to a Project
  2. they are added to a Milestone
  3. they have the lifecycle/frozen label
b0kky commented 7 months ago

Hello! @zeddit facing with same problem, and when i set AUTH_LDAP_ALLOW_SELF_SIGNED = True, and use ldaps url to AD, it working.

campi01 commented 6 months ago

Hello! @zeddit facing with same problem, and when i set AUTH_LDAP_ALLOW_SELF_SIGNED = True, and use ldaps url to AD, it working.

This helped me resolve the same issue in Superset after upgrading from 2.0.0 to 3.1.1, since Airflow and Superset both use Flask-AppBuilder for the LDAP authentication, and I was getting the same error before:

ERROR - {'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'}

mujohiddin commented 4 months ago

Hello! @zeddit facing with same problem, and when i set AUTH_LDAP_ALLOW_SELF_SIGNED = True, and use ldaps url to AD, it working.

This helped me resolve the same issue in Superset after upgrading from 2.0.0 to 3.1.1, since Airflow and Superset both use Flask-AppBuilder for the LDAP authentication, and I was getting the same error before:

ERROR - {'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'}

@zeddit did you solve this problem?