[bitnami/airflow] Error Occurs When Installing Chart via AWS CDK addHelmChart

[ebizboy]

Name and Version

bitnami/airflow 22.2.0

What architecture are you using?

amd64

What steps will reproduce the bug?

1. Attempting to install or update using CDK as follows:

   const airflowChart = cluster.addHelmChart("airflow-chart", {
     namespace: "airflow",
     repository: "https://charts.bitnami.com/bitnami",
     chart: "airflow",
     release: "airflow",
     version: "22.2.0",
     values: yaml.parse(
       fs.readFileSync(
         "manifests/bitnami-airflow-helm-values.yaml",
         "utf8"
       )
     ),
   });

   Are you using any custom parameters or values?

An error occurs regardless of the values

What is the expected behavior?

install or upgrade succeeds

What do you see instead?

DevAirflowStack failed: Error: The stack named DevAirflowStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Error: b'Release "airflow" does not exist. Installing it now.
Error: invalid_reference: invalid tag

Logs: /aws/lambda/DevAirflowStack-DevAirflowStackdev-Handler886CB40

    at invokeUserFunction (/var/task/framework.js:2:6)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async onEvent (/var/task/framework.js:1:369)
    at async Runtime.handler (/var/task/cfn-response.js:1:1837) (RequestId: 4bd8ca1d-c514-47eb-ab29-de01d207f3d8)

Additional information

I’m not sure whether this issue is related to the Airflow Helm chart or CDK, but there were no problems with chart version 21.x.x. The error occurs starting from version 22.x.x.

[juan131]

Hi @ebizboy

Could you share the bitnami-airflow-helm-values.yaml you're using? The values might be incompatible with Airflow 22.x, please note breaking changes were introduced on this version as it's explained below:

• https://github.com/bitnami/charts/tree/main/bitnami/airflow#upgrading

[ebizboy]

@juan131

Even if I try installing without the bitnami-airflow-helm-values.yaml, the error still occurs, so it seems that you don’t need to worry about the custom values.

Installing using helm install works without any issues. Could it be a problem with CDK?

[juan131]

Hi @ebizboy

I'm not familiar with CDK, do you know if it has any incompatibility with Helm hooks?

On version 21.9.0 (see https://github.com/bitnami/charts/commit/1a7d623709ba680e7d049e81bddc44d9115f5f1f) we added support for setting up the database on a K8s job that uses Helm hooks. You can disable this feature by setting setupDBJob.enabled to false and retry.

[primeos-work]

The invalid_reference: invalid tag error might be fixable by updating repository to oci://registry-1.docker.io/bitnamicharts (s. my comment here: https://github.com/bitnami/charts/issues/30582#issuecomment-2494545610).

[ebizboy]

@primeos-work

Changing the repo address to oci://registry-1.docker.io/bitnamicharts/airflow as you suggested resolved the issue. Thank you.

[ebizboy]

@juan131

In version 22.2.0, after setting setupDBJob.enabled to false, the airflow-scheduler still depends on wait-for-db-migrations, and only airflow-web no longer depends on it. Is this the expected behavior?

Following @primeos-work's suggestion above, I changed the repository URL to oci://registry-1.docker.io/bitnamicharts/airflow, and the CDK invalid tag error no longer occurs. The issue has been resolved. Thank you all.

[juan131]

Yes @ebizboy it's the expected behavior given it's the airflow-web will be in charged of running the db migrations

I changed the repository URL to oci://registry-1.docker.io/bitnamicharts/airflow, and the CDK invalid tag error no longer occurs. The issue has been resolved

Awesome! I proceed to close the issue