thesuperzapper
commented
3 years ago This is very likely a duplicate of https://github.com/airflow-helm/charts/issues/169
Closed klalafaryan closed 3 years ago
thesuperzapper
commented
3 years ago This is very likely a duplicate of https://github.com/airflow-helm/charts/issues/169
klalafaryan
commented
3 years ago @thesuperzapper To be honest, I am not sure if it is related to #169.
The problem here is with 2.0.1 image, and it pip install requires --use-feature=2020-resolver which is currently missing.
Did I miss anything ?
thesuperzapper
commented
3 years ago I dont think thats an error (just a notice from pip that things might fail in future versions), please copy the logs corresponding to the actual failure.
klalafaryan
commented
3 years ago Here are the logs, airflow-web pod is failing:
kubectl logs -f pods/airflow-web-7d675f8c6f-lwhjx -n airflow airflow-web
/opt/python/site-packages/airflow/configuration.py:332 DeprecationWarning: The logging_level option in [core] has been moved to the logging_level option in [logging] - the old setting has been used, but please update your config.
/opt/python/site-packages/airflow/configuration.py:332 DeprecationWarning: The remote_logging option in [core] has been moved to the remote_logging option in [logging] - the old setting has been used, but please update your config.
/opt/python/site-packages/airflow/configuration.py:332 DeprecationWarning: The remote_base_log_folder option in [core] has been moved to the remote_base_log_folder option in [logging] - the old setting has been used, but please update your config.
/opt/python/site-packages/sqlalchemy/orm/relationships.py:3441 SAWarning: relationship 'DagRun.serialized_dag' will copy column serialized_dag.dag_id to column dag_run.dag_id, which conflicts with relationship(s): 'TaskInstance.dag_run' (copies task_instance.dag_id to dag_run.dag_id), 'DagRun.task_instances' (copies task_instance.dag_id to dag_run.dag_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. The 'overlaps' parameter may be used to remove this warning. (Background on this error at: http://sqlalche.me/e/14/qzyx)
/opt/python/site-packages/sqlalchemy/orm/relationships.py:3441 SAWarning: relationship 'SerializedDagModel.dag_runs' will copy column serialized_dag.dag_id to column dag_run.dag_id, which conflicts with relationship(s): 'TaskInstance.dag_run' (copies task_instance.dag_id to dag_run.dag_id), 'DagRun.task_instances' (copies task_instance.dag_id to dag_run.dag_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. The 'overlaps' parameter may be used to remove this warning. (Background on this error at: http://sqlalche.me/e/14/qzyx)
/home/airflow/.local/lib/python3.8/site-packages/azure/cosmos/session.py:186 SyntaxWarning: "is not" with a literal. Did you mean "!="?
[2021-04-22 12:46:09,330] {providers_manager.py:295} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
[2021-04-22 12:46:09,982] {providers_manager.py:295} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-04-22 12:46:10,230] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/opt/python/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/opt/python/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/opt/python/site-packages/airflow/utils/cli.py", line 89, in wrapper
return f(*args, **kwargs)
File "/opt/python/site-packages/airflow/cli/commands/webserver_command.py", line 360, in webserver
app = cached_app(None)
File "/opt/python/site-packages/airflow/www/app.py", line 135, in cached_app
app = create_app(config=config, testing=testing)
File "/opt/python/site-packages/airflow/www/app.py", line 113, in create_app
init_appbuilder(flask_app)
File "/opt/python/site-packages/airflow/www/extensions/init_appbuilder.py", line 46, in init_appbuilder
AirflowAppBuilder(
File "/opt/python/site-packages/flask_appbuilder/base.py", line 148, in __init__
self.init_app(app, session)
File "/opt/python/site-packages/flask_appbuilder/base.py", line 202, in init_app
self.sm = self.security_manager_class(self)
File "/opt/python/site-packages/airflow/www/security.py", line 160, in __init__
super().__init__(appbuilder)
File "/opt/python/site-packages/flask_appbuilder/security/sqla/manager.py", line 51, in __init__
super(SecurityManager, self).__init__(appbuilder)
File "/opt/python/site-packages/flask_appbuilder/security/manager.py", line 250, in __init__
from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'@thesuperzapper Any ideas ?
klalafaryan
commented
3 years ago @thesuperzapper I have fixed the problem with the following:
extraPipPackages:
## the following configs require Flask-AppBuilder 3.2.0 (or later)
- "Flask-AppBuilder~=3.2.0"
## the following configs require Authlib
- "Authlib~=0.15.3"
- "apache-airflow[google_auth,jdbc,kubernetes,postgres,s3,ssh,databricks]==2.0.1"I think we can close the issue since this is not related to the helm chart at all, but I think the issue should be fixed in apache-airflow[google_auth].
thesuperzapper
commented
3 years ago @klalafaryan do you want to raise it on the apache/airflow repo, if it's not already there?
klalafaryan
commented
3 years ago @thesuperzapper created an issue in the apache/airflow project.
potiuk
commented
3 years ago @thesuperzapper I have fixed the problem with the following:
extraPipPackages: ## the following configs require Flask-AppBuilder 3.2.0 (or later) - "Flask-AppBuilder~=3.2.0" ## the following configs require Authlib - "Authlib~=0.15.3" - "apache-airflow[google_auth,jdbc,kubernetes,postgres,s3,ssh,databricks]==2.0.1"I think we can close the issue since this is not related to the helm chart at all, but I think the issue should be fixed in
apache-airflow[google_auth].
It is related to helm chart and the way how extraPackages are added. When Airflow 2.0.2 got released, the way how extraPackages are used, caused the airflow to be reinstalled in a different site-package directory and it "overrides" the original location of Airlfow. So what you end-up with in the chart is an image with two versions of airflow installed in two different locations. This is chart problem. not airflow. I even commented on how this can be fixed, following the exact way how Airfow image is built: https://github.com/airflow-helm/charts/issues/169#issuecomment-824955814
klalafaryan
commented
3 years ago @thesuperzapper Any ideas ?
thesuperzapper
commented
3 years ago @klalafaryan can you confirm if your issue is fixed after version 8.0.9 of the chart?
klalafaryan
commented
3 years ago @thesuperzapper I have tried 8.1.0 version, and it is not fixed. Still I get following exception:
[2021-05-11 15:31:53,625] {providers_manager.py:295} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
[2021-05-11 15:31:54,014] {providers_manager.py:295} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-05-11 15:31:54,377] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 89, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/webserver_command.py", line 360, in webserver
app = cached_app(None)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/app.py", line 135, in cached_app
app = create_app(config=config, testing=testing)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/app.py", line 113, in create_app
init_appbuilder(flask_app)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/extensions/init_appbuilder.py", line 46, in init_appbuilder
AirflowAppBuilder(
File "/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/base.py", line 148, in __init__
self.init_app(app, session)
File "/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/base.py", line 202, in init_app
self.sm = self.security_manager_class(self)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/security.py", line 160, in __init__
super().__init__(appbuilder)
File "/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/security/sqla/manager.py", line 51, in __init__
super(SecurityManager, self).__init__(appbuilder)
File "/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/security/manager.py", line 250, in __init__
from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'@klalafaryan the issue is because you are not installing the Authlib python package, please review the docs for OAUTH:
https://github.com/airflow-helm/charts/tree/main/charts/airflow#how-to-authenticate-airflow-users-with-ldapoauth
thesuperzapper
commented
3 years ago @klalafaryan did you resolve your issue?
rashi-psg
commented
3 years ago Facing the same issue with chart 8.4.0. I am using a custom image based on apache/airflow:2.0.2-python3.8 and installed the required packages inside it. Have authlib also in the image. But the webserver pod is failing with the error
[2021-07-12 07:31:13,978] {dagbag.py:451} INFO - Filling up the DagBag from /dev/null
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in
On adding Authlib to extraPipPackages is giving the same error.
thesuperzapper
commented
3 years ago @rashi-psg, what container is that (webserver, git-sync, etc)?
Also, what packages are you installing?
rashi-psg
commented
3 years ago Error is in webserver container. Installing the following because it is somehow not picking the packages present in image extraPipPackages:
thesuperzapper
commented
3 years ago @rashi-psg you mention that you are already installing Authlib in your container image, so I have a few questions:
authlib?
docker run -it ... it locally, and importing authlib in a python shell)pullPolicy: IfNotPresent, make sure you have renamed the image tag, so it knows to pull it again)
stale[bot]
commented
3 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
What is the bug?
With above extraPipPackages,
install-pip-packagescontainer fails with following exception:What is your Kubernetes Version?:
What is your Helm version?: