search

fluxcd / image-automation-controller

GitOps Toolkit controller that patches container image tags in Git
https://fluxcd.io
Apache License 2.0
178 stars 72 forks source link

Issues with applying policies.. walking path for files on AKS #744

Closed rasmusviben closed 2 months ago

rasmusviben commented 2 months ago

I'm setting up Flux on Azure with AKS, but I'm running into an issue with the ImageUpdateAutomation configuration that I can't wrap my head around.

Flux version 

flux: v2.3.0
distribution: flux-v2.3.0
helm-controller: v1.0.1
image-automation-controller: v0.38.0
image-reflector-controller: v0.32.0
kustomize-controller: v1.3.0
notification-controller: v1.3.0
source-controller: v1.3.0

Details from flux-system event log on AKS including the ERROR in message: failed to apply policies: walking path for files: lstat /tmp/flux-system-flux-system2486395273/workloads: no such file or directory - It looks like it writes flux-system twice in the path? I would expect it only to be /tmp/flux-system2486395273/workloads. And no matter what I change in the image-automation.yaml, it still wont shorten the path as expected. :/

{
    "kind": "Event",
    "apiVersion": "v1",
    "metadata": {
        "name": "image-automation.17f2af366334c67c",
        "namespace": "flux-system",
        "uid": "114fbbfc-a36f-47d1-aac0-3d8332038557",
        "resourceVersion": "53732",
        "creationTimestamp": "2024-09-06T14:50:55Z",
        "managedFields": [
            {
             xxx
        ]
    },
    "involvedObject": {
        "kind": "ImageUpdateAutomation",
        "namespace": "flux-system",
        "name": "image-automation",
        "uid": "757bf119-5c47-489d-997f-513389d12aeb",
        "apiVersion": "image.toolkit.fluxcd.io/v1beta2",
        "resourceVersion": "3601997"
    },
    "reason": "UpdateFailed",
    "message": "failed to apply policies: walking path for files: lstat /tmp/flux-system-flux-system2486395273/workloads: no such file or directory",
    "source": {
        "component": "image-automation-controller"
    },
    "firstTimestamp": "2024-09-06T14:50:55Z",
    "lastTimestamp": "2024-09-06T14:50:55Z",
    "count": 1,
    "type": "Warning",
    "eventTime": null,
    "reportingComponent": "image-automation-controller",
    "reportingInstance": ""
}

image-automation.yaml

---
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: flux-system
  namespace: flux-system
spec:
  git:
    checkout:
      ref:
        branch: master
    commit:
      author:
        email: flux@azure-no-reply.com
        name: flux
      messageTemplate: '[Upgrade] {{range .Updated.Images}}{{println .}}{{end}}'
    push:
      branch: master
  interval: 1m0s
  sourceRef:
    kind: GitRepository
    name: flux-system
    namespace: flux-system
  update:
    path: ./clusters/prod
    strategy: Setters

Flux check

► checking prerequisites
✔ Kubernetes 1.30.3 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.3.0
✔ bootstrapped: true
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.0.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.38.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.32.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.3.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.3.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.3.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ fluxconfigs.clusterconfig.azure.com/v1alpha1
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

Git repo setup

clusters/
  cluster_name/
      flux-system/
           gotk-components.yaml
           gotk-sync.yaml
           image-automation.yaml -> problem child. :) 
           kustomization.yaml
      helm-sources/
      images/
      namespaces/
      releases/
      workloads/
makkes commented 2 months ago

The error occurs with an ImageUpdateAutomation called image-automation in the flux-system Namespace while the manifest you pasted above is called flux-system, so I suppose there's another ImageUpdateAutomation in your cluster that has a .spec.update.path of ./workloads. Check the existing objects with

flux get image update
makkes commented 2 months ago

It looks like it writes flux-system twice in the path?

This is expected because this path is comprised of the source object's Namespace and and name.

rasmusviben commented 2 months ago

I changed my image-automation.yaml and pushed to branch master, but it didnt get updated automatically apparently - I'm pretty new to Flux, so this might be as expected. :)

The issue was fixed when i applied the updated file manually with kubectl.