search

industrydive / fileflow

Airflow plugin to transfer arbitrary files between operators
http://fileflow.readthedocs.io/en/latest/
Apache License 2.0
78 stars 21 forks source link

(closes #8) Adding full fileflow example to docs. #9

Closed lauralorenz closed 7 years ago

lauralorenz commented 7 years ago

Closes #8. Adds an example DAG with comments and how to run it that showcases the fileflow file backed storage.

To build the docs locally: given you installed fileflow with python setup.py install you can add the docs libraries you need with pip install fileflow[docs] and run make docs from inside the docs/ folder (where the Makefile exists)

I also included in this PR 1) setting it up so that there is an extra_requires for the docs builds (so that testing the rendering is possible above lol) 2) flake8-ing

Must trigger a new docs build on readthedocs after this is merged ❗️

lauralorenz commented 7 years ago

I'd still like to include a luigi vs. airflow/fileflow example, but could also open that separately if we want to merge this in now.