apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.71k stars 14.21k forks source link

example_twitter_dag.py cannot import name HiveOperator #1446

Closed dennisobrien closed 8 years ago

dennisobrien commented 8 years ago

Dear Airflow Maintainers,

Environment

Before I tell you about my issue, let me describe my Airflow environment:

Please fill out any appropriate fields:

After installing airflow from the tag 'airbnb_1.7.1rc3' I see this error on the admin/ page:

Broken DAG: [/home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py] cannot import name HiveOperator

Following the imports, I think it is due to this line in airflow/hooks/hive_hooks.py trying to import unicodecsv: https://github.com/airbnb/airflow/blame/master/airflow/hooks/hive_hooks.py#L19

My requirements.txt does not include airflow[hive] so it did not pull in these extra dependencies.

[2016-04-27 23:24:01,410] {models.py:250} ERROR - Failed to import: /home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py
Traceback (most recent call last):
  File "/home/airflow/venv/src/airflow/airflow/models.py", line 247, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py", line 26, in <module>
    from airflow.operators import BashOperator, HiveOperator, PythonOperator
ImportError: cannot import name HiveOperator

Reproducing the Issue

Here is how you can reproduce this issue on your machine.

bolkedebruin commented 8 years ago

Please use Apache Jira (https://issues.apache.org/jira/browse/AIRFLOW). GitHub issues are closed.

For your particular issue make sure you have installed the dependencies for Hive.