bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
8.85k stars 9.14k forks source link

can't find the configuration file **airflow.cfg** #8577

Closed alexisaraya closed 2 years ago

alexisaraya commented 2 years ago

Which chart:

Airflow 11.1.13

https://github.com/bitnami/charts/blob/master/bitnami/airflow/

Describe the bug

I can't find the configuration file airflow.cfg in webserver pod

To Reproduce Steps to reproduce the behavior:

  1. Go to '...' login to the webserver pod via cli console
  2. Go to folder /opt/bitnami/airflow
  3. I can't find the airflow folder inside folder
  4. I can't find configuration file airflow.cfg

Expected behavior you should have the folder and the configuration file

Version of Helm and Kubernetes:

- Output of `kubectl version`:

1.2



**Additional context**
I need to enable azure keyvault, as backend secret
carrodher commented 2 years ago

The file is present at /opt/bitnami/airflow, see

$ kubectl exec -it airflow-web-ddcd8f4fc-2r27p -n carlosrh-airflow -- ls -la /opt/bitnami/airflow
total 104
drwxrwxr-x 1 root root  4096 Jan  5 17:12 .
drwxrwxr-x 1 root root  4096 Dec 28 17:43 ..
-rw-r--r-- 1 1001 root 43458 Jan  5 17:11 airflow.cfg
drwxrwxr-x 2 root root  4096 Dec 28 17:44 dags
lrwxrwxrwx 1 1001 root    21 Jan  5 17:12 data -> /bitnami/airflow/data
drwxrwxr-x 1 root root  4096 Dec 21 18:52 licenses
drwxrwxr-x 1 root root  4096 Dec 28 17:44 logs
-rw-r--r-- 1 1001 root    13 Jan  5 17:11 nss_group
-rw-r--r-- 1 1001 root    57 Jan  5 17:11 nss_passwd
drwxrwxr-x 1 root root  4096 Jan  5 17:12 tmp
drwxrwxr-x 1 root root  4096 Dec 21 18:52 venv
-rw-rw-r-- 1 1001 root  4695 Jan  5 17:11 webserver_config.py
alexisaraya commented 2 years ago

Hi @carrodher The airflow folder does not exist in the pod.

$kubectl exec -it airflowanalytics-qa-web-649986c4b9-bhsjk -n airflowanalytics-qa -- ls -la /opt/bitnami/

Defaulted container "sync-repositories" out of: sync-repositories, airflow-web, clone-repositories (init) total 28 drwxrwxr-x 1 root root 4096 Nov 15 06:48 . drwxrwxr-x 1 root root 4096 Nov 15 06:48 .. -rw-r--r-- 1 root root 435 Nov 15 06:48 .bitnami_components.json drwxr-xr-x 1 root root 4096 Aug 17 00:26 common drwxr-xr-x 6 root root 4096 Aug 17 00:26 git drwxrwxr-x 2 root root 4096 Nov 15 06:48 licenses drwxrwxr-x 1 root root 4096 Nov 15 06:48 scripts

carrodher commented 2 years ago

It's really weird, can you share with us the steps you're using to deploy the Helm chart? Are you performing any customization via values?

alexisaraya commented 2 years ago

Hi @carrodher , We are using terraform, for azure cloud deployment. It is running on an AKS The repository we are using is https://marketplace.azurecr.io/helm/v1/repo",

using postgres and external redis as a service in azure, and all parameters are sent

example Parametes:

externalDatabase.host
externalDatabase.port
externalDatabase.database
externalDatabase.user
externalDatabase.password

The service is working fine, including ingress, but I can't find the configuration files in the Pods. I have not found the way to get the secret backend via parameter either.

If you can guide me in any way

carrodher commented 2 years ago

I just tried using the chart from the azure-marketplace repo and everything looks fine:

$ helm repo add azure-marketplace https://marketplace.azurecr.io/helm/v1/repo
"azure-marketplace2" has been added to your repositories

$ helm install airflow azure-marketplace/airflow --namespace airflow --create-namespace
NAME: airflow
LAST DEPLOYED: Tue Jan 11 07:30:57 2022
NAMESPACE: airflow
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: airflow
CHART VERSION: 11.2.0
APP VERSION: 2.2.3

** Please be patient while the chart is being deployed **1. Get the Airflow URL by running:

  echo URL  : http://127.0.0.1:8080
  kubectl port-forward --namespace airflow svc/airflow 8080:8080

2. Get your Airflow login credentials by running:
  export AIRFLOW_PASSWORD=$(kubectl get secret --namespace "airflow" airflow -o jsonpath="{.data.airflow-password}" | base64 --decode)
  echo User:     user

$ kubectl exec -it airflow-scheduler-867b7dd686-lpgmj -n airflow -- ls -la /opt/bitnami/airflow/airflow.cfg
-rw-r--r-- 1 1001 root 43448 Jan 11 07:41 /opt/bitnami/airflow/airflow.cfg

$ kubectl exec -it airflow-web-7cb5879c57-67s7z -n airflow -- ls -la /opt/bitnami/airflow/airflow.cfg
-rw-r--r-- 1 1001 root 43458 Jan 11 07:37 /opt/bitnami/airflow/airflow.cfg

$ kubectl exec -it airflow-worker-0 -n airflow -- ls -la /opt/bitnami/airflow/airflow.cfg
-rw-r--r-- 1 1001 root 43447 Jan 11 07:43 /opt/bitnami/airflow/airflow.cfg

Per the above logs, the airflow.cfg file is present in all the containers deployed in the different airflow-* pods. Can you try installing it from scratch in a new namespace as shown in the above example?

github-actions[bot] commented 2 years ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

alexisaraya commented 2 years ago

hi @carrodher , I confirm that using comand via cli works and the configuration file appears, maybe the problem is generated by the use of terraform and sending the parameters.