zitadel / zitadel-charts

This repository contains Helm charts for running ZITADEL in Kubernetes
https://zitadel.com
Apache License 2.0
76 stars 59 forks source link

Switch from hooks to using initContainers to wait for dependencies, such as databases, to be ready before initialization. #280

Open lknite opened 1 week ago

lknite commented 1 week ago

Preflight Checklist

Describe your problem

Per discussion: https://github.com/zitadel/zitadel-charts/discussions/270#discussioncomment-11365680

Summary: zitadel currently uses hooks to wait for a database to be ready, but database will not deploy because zitadel is using hooks, creating a chicken/egg scenario when using helm dependencies.

e.g. Chart.yaml:

...
dependencies:
- name: cockroachdb
  version: 11.2.1
  #version: 14.0.5
  repository: https://charts.cockroachdb.com/
- name: zitadel
  version: 8.5.0
  repository: https://charts.zitadel.com

Describe your ideal solution

Either the application itself could watch for the ready status of dependencies, or initContainers could be used to wait for ready status.

Example use case: I use an initContainer with sealed-secrets because I use external-secrets to provide a "common" secret shared among clusters, external-secrets provides the secret but I need sealed-secrets to wait to start until after the secret is created, because sealed-secrets will not notice the secret if its created after the pod is started.

Version

latest

Environment

Self-hosted

Additional Context

No response

lknite commented 1 week ago

Kubernetes API Reference: https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/

When learning kubeapi, it can be useful to pass --v=8 to any kubectl command, this let's you see what the full url was used to perform the action. e.g. k get pods --v=8

Here's an example to wait for a secret, maybe you could use something similar: (for this to work you have to grant the default service account the necessary RBAC to get/list secrets)

initContainers:
- name: wait-for-secret
  image: alpine/curl
  imagePullPolicy: Always
  command: ['sh', '-c']
  args:
  - |
    # Point to the internal API server hostname
    APISERVER=https://kubernetes.default.svc

    # Path to ServiceAccount token
    SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount

    # Read this Pod's namespace
    NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)

    # Read the ServiceAccount bearer token
    TOKEN=$(cat ${SERVICEACCOUNT}/token)

    # Reference the internal certificate authority (CA)
    CACERT=${SERVICEACCOUNT}/ca.crt

    # Explore the API with TOKEN
    while [ 200 -ne $(curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" \
      -X GET ${APISERVER}/api/v1/namespaces/sealed-secrets/secrets/sealed-secrets-shared-key \
      --head \
      --silent \
      --output /dev/null \
      --write-out '%{http_code}') ]
    do
      echo "waiting for secret: sealed-secrets-shared-key"
      sleep 5
    done
    echo "waiting for secret: sealed-secrets-shared-key"
    echo "  detected"