industrydive / fileflow

Airflow plugin to transfer arbitrary files between operators
http://fileflow.readthedocs.io/en/latest/
Apache License 2.0
78 stars 21 forks source link

Airflow throws a fit of the fileflow section doesn't exist #5

Closed MiriamSexton closed 7 years ago

MiriamSexton commented 7 years ago

If for some reason you're running the code without setting an airflow.cfg that contains a fileflow section, airflow freaks out.

For example, trying to build the docs in a new checkout of fileflow produced this:


~/fileflow/docs/source/task_runners.rst:7: WARNING: autodoc: failed to import module u'fileflow.task_runners.task_runner'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/miriamsexton/.venv/fileflow_tests/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 547, in import_object
    __import__(self.modname)
  File "~/fileflow/fileflow/task_runners/__init__.py", line 1, in <module>
    from .task_runner import TaskRunner
  File "~/fileflow/fileflow/task_runners/task_runner.py", line 13, in <module>
    from fileflow.storage_drivers import get_storage_driver
  File "~/fileflow/fileflow/storage_drivers/__init__.py", line 4, in <module>
    from .. import configuration
  File "~/fileflow/fileflow/configuration.py", line 12, in <module>
    airflow_configuration.set('fileflow', 'environment', 'production')
  File "/Users/miriamsexton/.venv/fileflow_tests/lib/python2.7/site-packages/airflow/configuration.py", line 691, in set
    return conf.set(section, option, value)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 396, in set
    raise NoSectionError(section)
NoSectionError: No section: 'fileflow'

Airflow is doing weird things with its configuration.