search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.19k stars 14.32k forks source link

Master (post 1.7.0) breaks the github_auth plugin #1303

Closed diranged closed 8 years ago

diranged commented 8 years ago

Environment

We have the following setup in our airflow.cfg file:

[webserver]
...
authenticate = $WEBSERVER_AUTHENTICATE
auth_backend = $WEBSERVER_AUTH_BACKEND

[github_enterprise]
host = $GHE_HOST
client_id = $GHE_CLIENT_ID
client_secret = $GHE_CLIENT_SECRET
allowed_teams = $GHE_ALLOWED_TEAMS
oauth_callback_route = $GHE_OAUTH_CALLBACK_ROUTE

We are passing in the following environment variables:

# Create some default GHE Oauth Credential variables -- the secret/id need to
# be set when you run the container.
export GHE_HOST=${GHE_HOST:-git.domain.com}
export GHE_ALLOWED_TEAMS=${GHE_ALLOWED_TEAMS:-engineering}
export GHE_OAUTH_CALLBACK_ROUTE=${GHE_OAUTH_CALLBACK_ROUTE:-/ghe_oauth/callback}
export GHE_CLIENT_ID=...
export GHE_CLIENT_SECRET=...
export WEBSERVER_AUTHENTICATE=${WEBSERVER_AUTHENTICATE:-True}
export WEBSERVER_AUTH_BACKEND=${WEBSERVER_AUTH_BACKEND:-airflow.contrib.auth.backends.github_enterprise_auth}

If we run off of master (the SHA1 shown above), then we get this failure when we start the webserver:

+ exec airflow webserver -d
[2016-04-05 16:24:19,365] {__init__.py:36} INFO - Using executor CeleryExecutor
[2016-04-05 16:24:19,509] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2016-04-05 16:24:19,534] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/

[2016-04-05 16:24:20,068] {models.py:148} INFO - Filling up the DagBag from /app/dags
[2016-04-05 16:24:20,068] {models.py:226} INFO - Importing /app/dags/test.py
[2016-04-05 16:24:20,073] {models.py:318} INFO - Loaded DAG <DAG: test>
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 10, in <module>
    execfile(__file__)
  File "/src/airflow/airflow/bin/airflow", line 15, in <module>
    args.func(args)
  File "/src/airflow/airflow/bin/cli.py", line 335, in webserver
    app = cached_app(conf)
  File "/src/airflow/airflow/www/app.py", line 123, in cached_app
    app = create_app(config)
  File "/src/airflow/airflow/www/app.py", line 43, in create_app
    from airflow.www import views
  File "/src/airflow/airflow/www/views.py", line 73, in <module>
    login_required = airflow.login.login_required
AttributeError: 'module' object has no attribute 'login_required'

If we run the same exact environment, but off of the 1.7.0 tag, we get it working just fine:

+ exec airflow webserver -d
[2016-04-05 16:25:27,664] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2016-04-05 16:25:27,691] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-04-05 16:25:27,949] {__init__.py:36} INFO - Using executor CeleryExecutor
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/

[2016-04-05 16:25:28,324] {models.py:126} INFO - Filling up the DagBag from /app/dags
[2016-04-05 16:25:28,325] {models.py:204} INFO - Importing /app/dags/test.py
[2016-04-05 16:25:28,327] {models.py:296} INFO - Loaded DAG <DAG: test>
Starting the web server on port 8080 and host 0.0.0.0.
[2016-04-05 16:25:28,475] {_internal.py:87} INFO -  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
[2016-04-05 16:25:28,478] {_internal.py:87} INFO -  * Restarting with stat
[2016-04-05 16:25:28,854] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2016-04-05 16:25:28,883] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-04-05 16:25:29,153] {__init__.py:36} INFO - Using executor CeleryExecutor
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/

[2016-04-05 16:25:29,483] {models.py:126} INFO - Filling up the DagBag from /app/dags
[2016-04-05 16:25:29,485] {models.py:204} INFO - Importing /app/dags/test.py
[2016-04-05 16:25:29,487] {models.py:296} INFO - Loaded DAG <DAG: test>
Starting the web server on port 8080 and host 0.0.0.0.
[2016-04-05 16:25:29,632] {_internal.py:87} WARNING -  * Debugger is active!
[2016-04-05 16:25:29,634] {_internal.py:87} INFO -  * Debugger pin code: 818-054-806

Bug somewhere in master...

I havn't found where the bug is yet, but its clear that something changed in master after the 1.7.0 tag to break the github oauth module..

diranged commented 8 years ago

It looks like if we revert the github plugin itself and nothing else, the error goes away.

+# https://github.com/airbnb/airflow/issues/1303
+#
+# This is fixable by rolling back the github_enterprise_auth script to the
+# 1.7.0 version.
+RUN curl -o /src/airflow/airflow/contrib/auth/backends/github_enterprise_auth.py \
+            https://raw.githubusercontent.com/airbnb/airflow/1.7.0/airflow/contrib/auth/backends/github_enterprise_auth.py
+