stakater / Reloader

A Kubernetes controller to watch changes in ConfigMap and Secrets and do rolling upgrades on Pods with their associated Deployment, StatefulSet, DaemonSet and DeploymentConfig – [✩Star] if you're using it!
https://docs.stakater.com/reloader/
Apache License 2.0
7.41k stars 489 forks source link

[Question] Official documentation on using Reloader with a Cron Job ? #543

Open Hareshraman opened 11 months ago

Hareshraman commented 11 months ago

I have been following Reloader for the PR : https://github.com/stakater/Reloader/pull/486 Issue : Reloader Issue Support for Cron Job Is there a readme that's available ? for this change , would be really helpful

karl-johan-grahn commented 11 months ago

Why did you close https://github.com/stakater/Reloader/issues/542?

Hareshraman commented 11 months ago

Closed the other issue as Duplicate Issues were created.

Asking this question , tried using reloader by using version 1.0.42 in a k3s cluster.

`time="2023-10-09T09:46:09Z" level=info msg="Starting Controller to watch resource type: configMaps"
time="2023-10-09T09:46:09Z" level=info msg="created controller for: secrets"
time="2023-10-09T09:46:09Z" level=info msg="Starting Controller to watch resource type: secrets"
time="2023-10-09T09:49:34Z" level=error msg="Failed to list cronjobs cronjobs.batch is forbidden: User \"system:serviceaccount:ix:reloader-reloader\" cannot list resource \"cronjobs\" in API group \"batch\" in the namespace \"abc\""`

I have a configMap in the same namespace ' abc' if i were to change a key: value , the cron job would be triggered. The cron job has been annotated with

`annotations:
    configmap.reloader.stakater.com/reload: my-config`

my-config being the configmap in namespace abc.

Service account has been created "

Hareshraman commented 11 months ago

i stopped getting the log ensuring namespace selector was pointing to abc namespace, hence it was able to find -myconfig , but my cron job does have an annotations with

`annotations:
    configmap.reloader.stakater.com/reload: my-config`

Expected Behaviour Cron job is triggered by creating a new pod.

Seen behaviour -> no logs seen in reloader pods mentioning change in "abc" config map. Cron job not triggered. In the template i see that the resources that the service account have access to are deployment daemonsets and stateful set and is missing cronjob kind , is it expected that we add them to the template ? or is it expected to be cooked into the chart , here's the template Here's the tempalte


---

# Source: reloader/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    meta.helm.sh/release-namespace: "default"
    meta.helm.sh/release-name: "templates"
  labels:
    app: templates-reloader
    chart: "reloader-1.0.42"
    release: "templates"
    heritage: "Helm"
    app.kubernetes.io/managed-by: "Helm"
  name: templates-reloader
  namespace: default
---
# Source: reloader/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole
metadata:
  annotations:
    meta.helm.sh/release-namespace: "default"
    meta.helm.sh/release-name: "templates"
  labels:
    app: templates-reloader
    chart: "reloader-1.0.42"
    release: "templates"
    heritage: "Helm"
    app.kubernetes.io/managed-by: "Helm"
  name: templates-reloader-role
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
      - configmaps
    verbs:
      - list
      - get
      - watch
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "apps"
    resources:
      - deployments
      - daemonsets
      - statefulsets
    verbs:
      - list
      - get
      - update
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - deployments
      - daemonsets
    verbs:
      - list
      - get
      - update
      - patch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: reloader/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding
metadata:
  annotations:
    meta.helm.sh/release-namespace: "default"
    meta.helm.sh/release-name: "templates"
  labels:
    app: templates-reloader
    chart: "reloader-1.0.42"
    release: "templates"
    heritage: "Helm"
    app.kubernetes.io/managed-by: "Helm"
  name: templates-reloader-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: templates-reloader-role
subjects:
  - kind: ServiceAccount
    name: templates-reloader
    namespace: default
---
# Source: reloader/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    meta.helm.sh/release-namespace: "default"
    meta.helm.sh/release-name: "templates"
  labels:
    app: templates-reloader
    chart: "reloader-1.0.42"
    release: "templates"
    heritage: "Helm"
    app.kubernetes.io/managed-by: "Helm"
    group: com.stakater.platform
    provider: stakater
    version: v1.0.42
  name: templates-reloader
  namespace: default
spec:
  replicas: 1
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: templates-reloader
      release: "templates"
  template:
    metadata:
      labels:
        app: templates-reloader
        chart: "reloader-1.0.42"
        release: "templates"
        heritage: "Helm"
        app.kubernetes.io/managed-by: "Helm"
        group: com.stakater.platform
        provider: stakater
        version: v1.0.42
    spec:
      containers:
      - image: "ghcr.io/stakater/reloader:v1.0.42"
        imagePullPolicy: IfNotPresent
        name: templates-reloader

        ports:
        - name: http
          containerPort: 9090
        livenessProbe:
          httpGet:
            path: /live
            port: http
          timeoutSeconds: 5
          failureThreshold: 5
          periodSeconds: 10
          successThreshold: 1
          initialDelaySeconds: 10
        readinessProbe:
          httpGet:
            path: /metrics
            port: http
          timeoutSeconds: 5
          failureThreshold: 5
          periodSeconds: 10
          successThreshold: 1
          initialDelaySeconds: 10

        securityContext:
          {}
        args:
          - "--namespace-selector=abc"
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
      serviceAccountName: templates-reloader```
Hareshraman commented 11 months ago

I feel the cluster Role and the Role binding needs to be updated for a batch/job for the service account that is created with it

prakash-26790 commented 11 months ago

Would be good if document is updated with CronJob support.

I tried adding annotation for job like below

apiVersion: batch/v1
kind: Job
metadata:
  name: init-job
  annotations:
    reloader.stakater.com/auto: "true"

But seems not working.

Im using latest Helm chart version 1.0.44

Hareshraman commented 10 months ago

This was fixed in 1.0.45 and releases above

karl-johan-grahn commented 10 months ago

Documentation is still missing

RobCannon commented 8 months ago

Just noting that the HelmChart does not add the RBAC permissions to patch or update a Job/CronJob. It doesn't look like the controller (v1.0.52) is looking for and Jobs to update either. I would like to see this update a Job when a secret changes (and hopefully the Job will start again).

  - verbs:
      - list
      - get
    apiGroups:
      - batch
    resources:
      - cronjobs
  - verbs:
      - create
    apiGroups:
      - batch
    resources:
      - jobs
andreaswachs commented 2 days ago

Documentation is still missing

I just want to come in and do a ➕ 1️⃣ on this.

Since this behavior is not documented it caught us by surprise when our CronJobs were getting triggered when secrets got updated, which were otherwise supposed to run at fixed points in time and were quite expensive to run.