search

airflow-helm / charts

The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
https://github.com/airflow-helm/charts/tree/main/charts/airflow
Apache License 2.0
665 stars 476 forks source link

Support PersistentVolumeClaim retention feature #892

Closed Setsushin closed 1 month ago

Setsushin commented 2 months ago

Checks

Motivation

Sometimes we need extra workers for large tasks, but these tasks don't occur very often. We prefer not to keep workers on standby all the time, as this helps reduce costs.

Implementation

Ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention

Add .spec.persistentVolumeClaimRetentionPolicy property to the template file.

Are you willing & able to help?

thesuperzapper commented 1 month ago

@Setsushin we don't currently mount any PVCs on the worker StatefulSet by default.

Are you mounting one for some reason, and if so, can you confirm what you use this PVC for?

Setsushin commented 1 month ago

@thesuperzapper

Sorry, this is my misunderstanding. I found that the chart I used is not the community version but the official version. The official version defaults to using PVC and supports the configuration ofpersistentVolumeClaimRetentionPolicy.

Thanks for the reply and have a good day.