Flux Image Automation with cue

[kol-ratner]

hi @stefanprodan loving timoni, having an issue with automation of image updates by flux,

following along here: https://fluxcd.io/flux/guides/image-update/#configure-image-updates

and its not clear to me how i might be able to add the required marker comment to my deployment:

i have an included a naive attempt to add a comment to the image reference, it does not seem like the correct way to go about this given the following error from flux:

Deployment/platform/clarity-api-gateway dry-run failed: admission webhook "mutate.kyverno.svc-fail" denied the request: failed to add image information to the policy rule context: invalid image 'europe-west4-docker.pkg.dev/org-mgmt-398911/docker-getclarity-ai/platform/clarity-api-gateway-service:latest # {"$imagepolicy": "platform:clarity-api-gateway"}' (bad image: europe-west4-docker.pkg.dev/org-mgmt-398911/docker-getclarity-ai/platform/clarity-api-gateway-service:latest # {"$imagepolicy": "platform:clarity-api-gateway"}: invalid reference format)

my deployment.cue file is as follows

package templates

import (
    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
)

#Deployment: appsv1.#Deployment & {
    _config:    #Config
    _cmName:    string
    apiVersion: "apps/v1"
    kind:       "Deployment"
    metadata:   _config.metadata
    spec:       appsv1.#DeploymentSpec & {
        if !_config.autoscaling.enabled {
            replicas: _config.replicas
        }
        strategy: {
            type: "RollingUpdate"
            rollingUpdate: maxUnavailable: "50%"
        }
        selector: matchLabels: _config.selector.labels
        template: {
            metadata: {
                labels: _config.selector.labels
                if _config.podAnnotations != _|_ {
                    annotations: _config.podAnnotations
                }
                if !_config.monitoring.enabled {
                    annotations: {
                        "prometheus.io/scrape": "true"
                        "prometheus.io/port":   "\(_config.monitoring.port)"
                    }
                }
            }
            spec: corev1.#PodSpec & {
                serviceAccountName: _config.serviceAccount.name
                if _config.serviceAccount.create {
                    serviceAccountName: _config.metadata.name
                }

                containers: [
                    {
                        name:  _config.metadata.name
                        image: "\(_config.image.reference) # {\"$imagepolicy\": \"platform:clarity-api-gateway\"}"

                        imagePullPolicy: _config.imagePullPolicy
                        ports: [
                            {
                                name:          "http"
                                containerPort: _config.service.port
                                protocol:      "TCP"
                            },
                            if _config.monitoring.port != _config.service.port {
                                name:          "http-metrics"
                                containerPort: _config.monitoring.port
                                protocol:      "TCP"
                            },
                        ]
                        if _config.livenessProbe.enabled {
                            livenessProbe: {
                                httpGet: {
                                    path: _config.livenessProbe.path
                                    port: _config.livenessProbe.port
                                }
                                initialDelaySeconds: _config.livenessProbe.initialDelaySeconds
                                                periodSeconds: _config.livenessProbe.periodSeconds
                            }
                        }
                        if _config.readinessProbe.enabled {
                            readinessProbe: {
                                httpGet: {
                                    path: _config.readinessProbe.path
                                    port: _config.readinessProbe.port
                                }
                                initialDelaySeconds: _config.readinessProbe.initialDelaySeconds
                                                periodSeconds: _config.readinessProbe.periodSeconds
                            }
                        }
                        if _config.resources != _|_ {
                            resources: _config.resources
                        }
                        if _config.securityContext != _|_ {
                            securityContext: _config.securityContext
                        }
                        env: []
                        command: []
                        args: []
                        volumeMounts: [
                            if _config.configmap.enabled {
                                for c in _config.configmap.files {
                                    name:        c.volumeName
                                    mountPath:   c.mountPath
                                    subPathExpr: c.subPathExpr
                                }
                            },
                            // additional volumeMounts can be added here. for example:
                            // {
                            //  name:        "scratch"
                            //  mountPath:   "/home/scratch"
                            //  subPathExpr: "home"
                            // },
                        ]
                    },
                ]

                volumes: [
                    if _config.configmap.enabled {
                        for c in _config.configmap.files {
                            name: c.volumeName
                            configMap: {
                                name: _cmName
                                items: [{
                                    key:  c.key
                                    path: key
                                }]
                            }
                        }
                    },
                    // additional volumes can be added here, for example:
                    // {
                    //  name: "scratch"
                    //  emptyDir: {}
                    // },
                ]

                if _config.podSecurityContext != _|_ {
                    securityContext: _config.podSecurityContext
                }
                if _config.topologySpreadConstraints != _|_ {
                    topologySpreadConstraints: _config.topologySpreadConstraints
                }
                if _config.affinity != _|_ {
                    affinity: _config.affinity
                }
                if _config.tolerations != _|_ {
                    tolerations: _config.tolerations
                }
                if _config.imagePullSecrets != _|_ {
                    imagePullSecrets: _config.imagePullSecrets
                }
            }
        }
    }
}

[stefanprodan]

Flux can only update plain Kubernetes YAML which are stored in Git, adding a comment to objects in etcd will not do anything. Flux can't work with CUE files or Helm templates or any other format besides Kubernetes YAML. I'm also confused about what are you trying to achieve.

[kol-ratner]

hi @stefanprodan thanks for your response - to clarify my use case:

I am running timoni to build my k8s application manifests and am running fluxv2 to manage the cluster.

at present when we want to deploy a new application container image version we must manually update the image tag in the cue based values file.

i would like flux to watch the container registry and update the lastest deployed image. Ive been attempting to use flux's ImageRepository / ImagePolicy resources to handle this.

At present, I do understand that this wont work without flux's ImageUpdateAutomation resource, however flux cannot edit cue values files in my git repo.

Is it feasible to write a cue parsing tool which can update my cue values file and have it triggered by flux's ImagePolicy which is configured to watch the container registry... ?

Zooming out: do u have a general recommendation, given our current usage of timoni and flux on how to automatically update the deployed version of an application container image?

[stefanprodan]

Flux does not update things in-cluster, the image automation is pushing file changes to Git. Even if you would use Timoni Bundles in YAML format, you still need some other automation in CI to rerun the timoni push command.