During the rewriting and open sourcing of Amazon MWAA Airflow images we also want to change the way we set the configuration of Airflow. Currently, we provide a full airflow.cfg file. This might have potentially diverged from the defaults of Airflow, which might have changed throughout the versions. Thus, we would like to accept Airflow defaults unless we are opinionated about some configuration. This way if Airflow changes the default of some configuration, we also get that change when we support the new version.
Acceptance Criteria
Gather the list of Airflow configuration that Amazon MWAA is opinionated about or needs to setup for Airflow to work correctly in a MWAA environment (e.g. database configuration, SQS configuration, etc.)
Implement the necessary code to set those configuration. This has to be done via environment variables, as opposed to providing a large airlfow.cfg in which it is hard to tell what we have set and what is Airflow's default.
Overview
During the rewriting and open sourcing of Amazon MWAA Airflow images we also want to change the way we set the configuration of Airflow. Currently, we provide a full
airflow.cfgfile. This might have potentially diverged from the defaults of Airflow, which might have changed throughout the versions. Thus, we would like to accept Airflow defaults unless we are opinionated about some configuration. This way if Airflow changes the default of some configuration, we also get that change when we support the new version.Acceptance Criteria
airlfow.cfgin which it is hard to tell what we have set and what is Airflow's default.