stackabletech / airflow-operator

Stackable Operator for Apache Airflow
Other
21 stars 2 forks source link

Increase memory limit for git-sync container #467

Open dervoeti opened 1 month ago

dervoeti commented 1 month ago

A user reported that the git-sync containers crashes (OOM) when trying to sync a large repo with dags (>100 MB). We should either increase the memory limit (currently 64 Mi) or make this easier overridable. It's currently possible with podOverrides but they need to be added for both git-sync containers in each component:

...
  schedulers:
    podOverrides:
      spec:
        initContainers:
        - name: gitsync-0
          resources:
            limits:
              memory: 256Mi
              cpu: 100m
            requests:
              memory: 256Mi
              cpu: 200m
        containers:
        - name: gitsync-1
          resources:
            limits:
              memory: 256Mi
              cpu: 100m
            requests:
              memory: 256Mi
              cpu: 200m
  celeryExecutors:
    podOverrides:
...