Comparison with Meltano, Mara, Airflow and other ETL tools

[rufuspollock]

As a potential User of dataflows I want to understand how it compares to other tools so that I get what use cases it was designed for and why (or why not) i should use it (and also deepen respect for its creators because i know they know their stuff).

As an example of this done very well see VuePress https://vuepress.vuejs.org/guide/#why-not (short) and VueJS https://vuejs.org/v2/guide/comparison.html (long)

Tasks

• [ ] How is this similar and different to meltano. Is there anything we could contribute there or learn from?
• [ ] Ditto for mara - https://github.com/mara/data-integration (python)
• [ ] Ditto for Apache Beam and Google DataFlow (based on Beam) - https://beam.apache.org/

[augusto-herrmann]

As far as I understand, Apache Airflow is a tool for orchestration and scheduling tool that could be used together with dataflows.

You define what to do to the data in dataflows. You define when to do it, manage job dependencies, what to do in case of failure, etc. in Airflow.