puckel / docker-airflow

Docker Apache Airflow
Apache License 2.0
3.77k stars 542 forks source link

Logs endpoint broken due to 1.5.1 config #6

Closed bhumpert closed 9 years ago

bhumpert commented 9 years ago

Whenever I try to access the logs endpoint in the admin console, webserver has the following exception:

2015-09-29 18:12:58,901 - airflow.www.app - ERROR - Exception on /admin/airflow/log [GET]
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python2.7/dist-packages/flask_admin/base.py", line 68, in inner
    return self._run_view(f, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/flask_admin/base.py", line 359, in _run_view
    return fn(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/airflow/www/app.py", line 795, in log
    loc = loc.format(**locals())
KeyError: 'AIRFLOW_HOME'
2015-09-29 18:12:58,902 - tornado.access - ERROR - 500 GET /admin/airflow/log?task_id=run_this_last&dag_id=example_bash_operator&execution_date=2015-09-28T00:00:00 (192.168.99.1) 5.89ms

The reason for this error is that Airflow is trying to format a string with {AIRFLOW_HOME} in it and failing because the key AIRFLOW_HOME does not appear in locals. The root of this problem seems to be that the {AIRFLOW_HOME} key is in the docker-airflow config/airflow.cfg under base_log_folder. It's easy enough to fix this problem (by cloning the repo and modifying the config), but I didn't know if there was a step which I was missing that would correctly format the config in the process of bringing the containers up. Really the only step that I'm performing is docker-compose up -d.

puckel commented 9 years ago

Hi,

I didn't reproduced your issue. I saw that you GET /admin/AIRFLOW/log in your request, but the logs from airflow are accessible from /admin/log

curl -IL "http://172.17.0.46:8080/admin/log/?task_id=run_this_last&dag_id=example_bash_operator" master HTTP/1.1 200 OK Content-Length: 285335 Content-Type: text/html; charset=utf-8 Server: TornadoServer/4.2.1

Try to pull the last version of the stask and re-run docker-compose

bhumpert commented 9 years ago

Those logs at /admin/log/ are very non-informative. You can get a more informative log at the route that I posted. You can navigate there in the gui by going to the graph of any DAG and clicking on a task, then "view log".

puckel commented 9 years ago

Hi,

I've updated the airflow.cfg with full path and I openned an issue in airflow/airbnb https://github.com/airbnb/airflow/issues/484

Thanks.

Puckel.