naani10
commented
1 year ago Dear community,
Can any one help me with this issue. Is there any workaround to avoid this errors?
Thanks
Closed naani10 closed 11 months ago
naani10
commented
1 year ago Dear community,
Can any one help me with this issue. Is there any workaround to avoid this errors?
Thanks
PankushSharma
commented
1 year ago Yeah we are also facing the same issue. Anyone able to find workaround?
crotielwf
commented
1 year ago Please check your Airflow version as there might be several deprecated variables that need to be updated. You can verify the variables using your Airflow. It is recommended to review the changes made after version 2.3.0, as there have been significant updates.
You can find more information on the official Apache Airflow documentation: link.
thesuperzapper
commented
1 year ago @naani10 @PankushSharma @crotielwf I have a suspicion that the issue is related to the webserver_config.py.
Are you using a custom webserver_config.py (which is configured with the web.webserverConfig.* values)?
If so, can you share it here?
If you are not using a custom webserver_config.py, I bet that our default one is not working properly.
Can try setting the webserver config to a slightly different default by using the values:
web:
webserverConfig:
stringOverride: |
from airflow.www.fab_security.manager import AUTH_DB
AUTH_TYPE = AUTH_DB@naani10 actually looking at your logs, the issue is in your custom /opt/airflow/keycloak_auth.py file.
The issue is the same as this user's problem (https://github.com/apache/airflow/issues/27138), where they were importing from flask_appbuilder.security.sqla.models import User, but the correct User module for airflow 2.3.0+ is from airflow.www.fab_security.sqla.models import User.
naani10
commented
1 year ago Hi @thesuperzapper please find the webserver_config.py below.
import os
from flask_appbuilder.security.manager import AUTH_DB
# from flask_appbuilder.security.manager import AUTH_LDAP
# from flask_appbuilder.security.manager import AUTH_OAUTH
# from flask_appbuilder.security.manager import AUTH_OID
# from flask_appbuilder.security.manager import AUTH_REMOTE_USER
basedir = os.path.abspath(os.path.dirname(__file__))
# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# ----------------------------------------------------
# AUTHENTICATION CONFIG
# ----------------------------------------------------
# For details on how to set up each of the following authentication, see
# http://flask-appbuilder.readthedocs.io/en/latest/security.html# authentication-methods
# for details.
# The authentication type
# AUTH_OID : Is for OpenID
# AUTH_DB : Is for database
# AUTH_LDAP : Is for LDAP
# AUTH_REMOTE_USER : Is for using REMOTE_USER from web server
# AUTH_OAUTH : Is for OAuth
AUTH_TYPE = AUTH_DB
# Uncomment to setup Full admin role name
AUTH_ROLE_ADMIN = 'Admin'
# Uncomment to setup Public role name, no authentication needed
AUTH_ROLE_PUBLIC = 'Op'
# Will allow user self registration
# AUTH_USER_REGISTRATION = True
# The recaptcha it's automatically enabled for user self registration is active and the keys are necessary
# RECAPTCHA_PRIVATE_KEY = PRIVATE_KEY
# RECAPTCHA_PUBLIC_KEY = PUBLIC_KEY
# Config for Flask-Mail necessary for user self registration
# MAIL_SERVER = 'smtp.gmail.com'
# MAIL_USE_TLS = True
# MAIL_USERNAME = 'yourappemail@gmail.com'
# MAIL_PASSWORD = 'passwordformail'
# MAIL_DEFAULT_SENDER = 'sender@gmail.com'
# The default user self registration role
# AUTH_USER_REGISTRATION_ROLE = "Public"
# When using OAuth Auth, uncomment to setup provider(s) info
# Google OAuth example:
# OAUTH_PROVIDERS = [{
# 'name':'google',
# 'token_key':'access_token',
# 'icon':'fa-google',
# 'remote_app': {
# 'api_base_url':'https://www.googleapis.com/oauth2/v2/',
# 'client_kwargs':{
# 'scope': 'email profile'
# },
# 'access_token_url':'https://accounts.google.com/o/oauth2/token',
# 'authorize_url':'https://accounts.google.com/o/oauth2/auth',
# 'request_token_url': None,
# 'client_id': GOOGLE_KEY,
# 'client_secret': GOOGLE_SECRET_KEY,
# }
# }]
# When using LDAP Auth, setup the ldap server
# AUTH_LDAP_SERVER = "ldap://ldapserver.new"
# When using OpenID Auth, uncomment to setup OpenID providers.
# example for OpenID authentication
# OPENID_PROVIDERS = [
# { 'name': 'Yahoo', 'url': 'https://me.yahoo.com' },
# { 'name': 'AOL', 'url': 'http://openid.aol.com/<username>' },
# { 'name': 'Flickr', 'url': 'http://www.flickr.com/<username>' },
# { 'name': 'MyOpenID', 'url': 'https://www.myopenid.com' }]
# ----------------------------------------------------
# Theme CONFIG
# ----------------------------------------------------
# Flask App Builder comes up with a number of predefined themes
# that you can use for Apache Airflow.
# http://flask-appbuilder.readthedocs.io/en/latest/customizing.html#changing-themes
# Please make sure to remove "navbar_color" configuration from airflow.cfg
# in order to fully utilize the theme. (or use that property in conjunction with theme)
APP_THEME = "bootstrap-theme.css" # default bootstrap
# APP_THEME = "amelia.css"
# APP_THEME = "cerulean.css"
# APP_THEME = "cosmo.css"
# APP_THEME = "cyborg.css"
# APP_THEME = "darkly.css"
# APP_THEME = "flatly.css"
# APP_THEME = "journal.css"
# APP_THEME = "lumen.css"
# APP_THEME = "paper.css"
# APP_THEME = "readable.css"
# APP_THEME = "sandstone.css"
# APP_THEME = "simplex.css"
# APP_THEME = "slate.css"
# APP_THEME = "solar.css"
# APP_THEME = "spacelab.css"
# APP_THEME = "superhero.css"
# APP_THEME = "united.css"
# APP_THEME = "yeti.css"@naani10 while you probably should update your webserver_config.py to use the new import locations for things like AUTH_LDAP, that is NOT the cause of your pods failing.
The issue is caused by your custom script at /opt/airflow/keycloak_auth.py. Clearly, it's not compatible with airflow since 2.3.0 for the reasons described in https://github.com/airflow-helm/charts/issues/747#issuecomment-1570748218.
You must either remove that script if you don't need it, or update it to support newer airflow.
mattefara
commented
1 year ago Hello everyone, I have the same issue but with a custom security manager. I'm using airflow 2.6.1 and helm chart 8.7.1 The code of webserver_config.py:
import os
from flask_appbuilder.security.manager import AUTH_DB
from my_modules.security import CustomSecurityManager
AUTH_TYPE = AUTH_DB
FAB_SECURITY_MANAGER_CLASS = CustomSecurityManagerand the code of the security manager:
from airflow.www.security import AirflowSecurityManager
class CustomSecurityManager(AirflowSecurityManager):
def auth_user_oauth(self, userinfo):
return super().auth_user_oauth(userinfo)If I comment FAB_SECURITY_MANAGER_CLASS and the import it works. Thanks
naani10
commented
11 months ago This was working after we updated below configs Thanks @thesuperzapper for support
web: webserverConfig: stringOverride: | from airflow.www.fab_security.manager import AUTH_DB AUTH_TYPE = AUTH_DB
Checks
User-Community Airflow Helm Chart.Chart Version
8.7.1
Kubernetes Version
Helm Version
Description
I am trying to install airflow 1.26.0 using 8.7.1 helm chart I am getting errors in sync user pod and airflow web pod airflow-sync-users-b8dc65576-j55b5.log airflow-web-7469d4449b-h46gd.log
Relevant Logs
Custom Helm Values
No response