Simplify - Githubissues

teamclairvoyant / apache-airflow-cloudera-csd

CSD for Apache Airflow

Apache License 2.0

20 stars 12 forks source link

Simplify #5

Closed razorsedge closed 5 years ago

razorsedge commented 5 years ago

Remove RabbitMQ parcel dependency.
Consolidate various scripts into one control.sh.
Dynamically build the airflow.cfg from the CM deployed Java properties file.
Run the Airflow daemons from the Cloudera Manager process directories just like the CDH daemons.
Provide Airflow gateway configs at /etc/airflow/conf/ .
Reorganize and set correct types/units on all properties.
Dynamically build database and Celery connection strings.

razorsedge commented 5 years ago

Requires Requires teamclairvoyant/apache-airflow-cloudera-parcel#4

thesuperzapper commented 5 years ago

I am also in the process of reworking this CSD with some of the same changes, but also lots of other stuff, including new parameters for newer Airflow, and more general cleanups.

One important change in my rewrite is no longer storing secure variables in the .cfg file.