Before I tell you about my issue, let me describe my Airflow environment:
Please fill out any appropriate fields:
Airflow version: (release number or git log --oneline | head -n 3)
airbnb_1.7.1rc3
Airflow components: (e.g. "webserver and scheduler with a postgres database and LocalExecutor")
Found in example_twitter_dag.py
Relevant airflow.cfg settings:
Python Version: (python --version)
Python 2.7.6
Operating System: (Windows version or uname -a)
Linux ip-172-31-46-134 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Running on Amazon EC2 Ubuntu 14.04 AMI
Dear Airflow Maintainers,
Environment
Before I tell you about my issue, let me describe my Airflow environment:
Please fill out any appropriate fields:
git log --oneline | head -n 3
) airbnb_1.7.1rc3airflow.cfg
settings:python --version
) Python 2.7.6uname -a
) Linux ip-172-31-46-134 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Running on Amazon EC2 Ubuntu 14.04 AMIpip freeze
orconda list
) $ pip freeze -e git://github.com/airbnb/airflow.git@f1ff65c8cde913fd9bfa8b21f83c7300b8b0effd#egg=airflow alembic==0.8.6 amqp==1.4.9 anyjson==0.3.3 Babel==1.3 backports.ssl-match-hostname==3.5.0.1 billiard==3.3.0.23 celery==3.1.23 certifi==2016.2.28 cffi==1.6.0 chartkick==0.4.2 croniter==0.3.12 cryptography==1.3.1 dill==0.2.5 docutils==0.12 enum34==1.1.4 Flask==0.10.1 Flask-Admin==1.4.0 Flask-Cache==0.13.1 Flask-Login==0.2.11 Flask-WTF==0.12 flower==0.8.4 funcsigs==0.4 future==0.15.2 futures==3.0.5 gunicorn==19.3.0 idna==2.1 ipaddress==1.0.16 itsdangerous==0.24 Jinja2==2.8 kombu==3.0.35 lockfile==0.12.2 Mako==1.0.4 Markdown==2.6.6 MarkupSafe==0.23 numpy==1.11.0 pandas==0.18.0 psycopg2==2.6.1 pyasn1==0.1.9 pycparser==2.14 Pygments==2.1.3 python-daemon==2.1.1 python-dateutil==2.5.3 python-editor==1.0 pytz==2016.4 redis==2.10.5 requests==2.9.1 setproctitle==1.1.9 six==1.10.0 SQLAlchemy==1.0.12 thrift==0.9.3 tornado==4.2 vertica-python==0.5.6 Werkzeug==0.11.9 WTForms==2.1Description of Issue
After installing airflow from the tag 'airbnb_1.7.1rc3' I see this error on the admin/ page:
Following the imports, I think it is due to this line in airflow/hooks/hive_hooks.py trying to import unicodecsv: https://github.com/airbnb/airflow/blame/master/airflow/hooks/hive_hooks.py#L19
My requirements.txt does not include airflow[hive] so it did not pull in these extra dependencies.
Reproducing the Issue
Here is how you can reproduce this issue on your machine.