airflowctl and GitHub usage

[eliezer41]

Hello there, I'm not sure if this is the right place to ask this question, but I was wondering what is the recommended way to use airflowctl, specifically settings.yaml file, if you want to put an airflow project into a GitHub repo.

I initially thought that you create the project in your dev environment using airflowctl init <project_name> ... as suggested in the docs. Then, you push the project to the GitHub repo, and once the project is ready for deployment, you just clone the repo into the production environment where you simply run airflowctl build provided settings.yaml already contains all the necessary variables for the project to work, but that would be a bad idea from a security perspective if you included connections with credentials in that file, which is my case.

So I don't know If I just have to add settings.yaml to .gitignore and create it manually after I clone the repo in production since I already have to do so with the .env file.

I will appreaciate any suggestion or guidance you can provide. Thanks in advance.

[kaxil]

you just clone the repo into the production environment where you simply run airflowctl build provided settings.yaml already contains all the necessary variables for the project to work, but that would be a bad idea from a security perspective if you included connections with credentials in that file, which is my case.

You can add your connections or variables via Environment Variables for Production usage.

airflowctl as it stands right now has mainly a single goal to make it easy to do local-dev :)

[kaxil]

For GitHub usage, yes you do an init and just store settings.yml file with the rest of the project in a Git repo. You should exclude the .env file which is where you "Secrets" should go.

For secrets in Production, you could also use Secrets backend

[eliezer41]

Thank you for the feedback!