fluxcd / flux2

Open and extensible continuous delivery solution for Kubernetes. Powered by GitOps Toolkit.
https://fluxcd.io
Apache License 2.0
6.47k stars 599 forks source link

Image Automation Controller only triggers on manual reconciliation. #5036

Open emanuele-leopardi opened 3 days ago

emanuele-leopardi commented 3 days ago

Describe the bug

I am experiencing an odd behavior of IAC. I expected IAC to be triggered by an update from an ImageRepository object, and to commit to my repository the updated tag. This only works if I manually trigger a reconciliation.

Steps to reproduce

  1. Install flux with image automation controller and image reflector controller
    flux bootstrap github \
    --token-auth \
    --owner=me \
    --repository=your-repo \
    --path=clusters/test \
    --components-extra=image-reflector-controller,image-automation-controller \
    --personal \
    --private \
    --branch flux
  2. Create the following
    apiVersion: source.toolkit.fluxcd.io/v1
    kind: HelmRepository
    metadata:
    name: httpbin
    namespace: httpbin
    spec:
    interval: 1m0s
    url: https://matheusfm.dev/charts
    ---
    apiVersion: helm.toolkit.fluxcd.io/v2
    kind: HelmRelease
    metadata:
    name: httpbin
    namespace: httpbin
    spec:
    chart:
    spec:
      chart: httpbin
      reconcileStrategy: ChartVersion
      sourceRef:
        kind: HelmRepository
        name: httpbin
        namespace: httpbin
    interval: 1m0s
    install:
    remediation:
      remediateLastFailure: true
      retries: 1
    upgrade:
    cleanupOnFail: true
    remediation:
      remediateLastFailure: true
      retries: -1
    maxHistory: 100
    values:
    image:
      repository: my-repo/httpbin # {"$imagepolicy": "httpbin:httpbin:name"}
      tag: test # {"$imagepolicy": "httpbin:httpbin:tag"}
      pullPolicy: Always
    ---
    apiVersion: image.toolkit.fluxcd.io/v1beta2
    kind: ImageRepository
    metadata:
    name: httpbin
    namespace: httpbin
    spec:
    image: my-repo/httpbin
    interval: 1m
    ---
    apiVersion: image.toolkit.fluxcd.io/v1beta2
    kind: ImagePolicy
    metadata:
    name: httpbin
    namespace: httpbin
    spec:
    imageRepositoryRef:
    name: httpbin
    filterTags:
    pattern: ''
    policy:
    alphabetical:
      order: asc
    ---
    apiVersion: image.toolkit.fluxcd.io/v1beta2
    kind: ImageUpdateAutomation
    metadata:
    name: httpbin
    namespace: httpbin
    spec:
    interval: 1m
    sourceRef:
    kind: GitRepository
    name: flux-system
    namespace: flux-system
    git:
    checkout:
      ref:
        branch: flux
    commit:
      author:
        email: fluxbot@keyless.io
        name: FluxBot
    update:
    path: ./clusters/test/apps/httpbin
    policySelector:
    matchLabels:
      app.kubernetes.io/instance: httpbin
      app.kubernetes.io/name: httpbin
    ---
    apiVersion: source.toolkit.fluxcd.io/v1
    kind: GitRepository
    metadata:
    name: flux-system
    namespace: flux-system
    spec:
    interval: 1m0s
    ref:
    branch: flux
    secretRef:
    name: flux-system
    url: your-repo.git

    The GitRepository above is the same repo used to bootstrap flux in the cluster. The cluster uses k3s on a 3 master + workers configuration. To test the behavior I chose policy.alphabetical.order: asc then I tagged the httpbin image as atest, btest, ctest etc. flux version is v.2.3.0

What happens is that ImageRepository correctly gets latest tags

  conditions:
    - lastTransitionTime: 2024-10-17T14:01:21Z
      message: "successful scan: found 5 tags"
      observedGeneration: 1
      reason: Succeeded
      status: "True"
      type: Ready
  lastHandledReconcileAt: 2024-10-15T15:25:09.946216846Z
  lastScanResult:
    latestTags:
      - ztest
      - vtest
      - utest
      - test
      - latest
    scanTime: 2024-10-17T16:32:46Z
    tagCount: 5

ImagePolicy matches the latest tag

status:
  conditions:
    - lastTransitionTime: 2024-10-17T14:01:21Z
      message: Latest image tag for 'my-repo/httpbin' updated
        from vtest to ztest
      observedGeneration: 2
      reason: Succeeded
      status: "True"
      type: Ready
  latestImage: my-repo/httpbin:ztest
  observedGeneration: 2
  observedPreviousImage: my-repo/httpbin:vtest

ImageUpdateAutomation says the repository is up to date but it doesn't push any update to the git repository

status:
  conditions:
    - lastTransitionTime: 2024-10-17T03:36:58Z
      message: repository up-to-date
      observedGeneration: 11
      reason: Succeeded
      status: "True"
      type: Ready
  lastAutomationRunTime: 2024-10-17T16:35:15Z
  lastHandledReconcileAt: 2024-10-17T14:14:27.636419221Z
  lastPushCommit: f850e802023aa075bae295a1d2e637f06d9e0b7b
  lastPushTime: 2024-10-17T14:14:29Z
  observedGeneration: 11

If I then run flux reconcile against ImageUpdateAutomation , then boom: image automation controller pushes the commit toward the repository image

I am probably missing something and I am hopeful that @stefanprodan will be able to give me the correct nudge to get this sorted

Expected behavior

I was expecting IAC to automatically push the commit to the repo without manual reconciliation

Screenshots and recordings

No response

OS / Distro

N/A

Flux version

flux: v2.3.0

Flux check

► checking prerequisites ✗ flux 2.3.0 <2.4.0 (new CLI version is available, please upgrade) ✗ Kubernetes version v1.24.17+k3s1 does not match >=1.28.0-0 ► checking version in cluster ✔ distribution: flux-v2.3.0 ✔ bootstrapped: true ► checking controllers ✔ helm-controller: deployment ready ► ghcr.io/fluxcd/helm-controller:v1.0.1 ✔ image-automation-controller: deployment ready ► ghcr.io/fluxcd/image-automation-controller:v0.38.0 ✔ image-reflector-controller: deployment ready ► ghcr.io/fluxcd/image-reflector-controller:v0.32.0 ✔ kustomize-controller: deployment ready ► ghcr.io/fluxcd/kustomize-controller:v1.3.0 ✔ notification-controller: deployment ready ► ghcr.io/fluxcd/notification-controller:v1.3.0 ✔ source-controller: deployment ready ► ghcr.io/fluxcd/source-controller:v1.3.0 ► checking crds ✔ alerts.notification.toolkit.fluxcd.io/v1beta3 ✔ buckets.source.toolkit.fluxcd.io/v1beta2 ✔ gitrepositories.source.toolkit.fluxcd.io/v1 ✔ helmcharts.source.toolkit.fluxcd.io/v1 ✔ helmreleases.helm.toolkit.fluxcd.io/v2 ✔ helmrepositories.source.toolkit.fluxcd.io/v1 ✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2 ✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2 ✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2 ✔ kustomizations.kustomize.toolkit.fluxcd.io/v1 ✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2 ✔ providers.notification.toolkit.fluxcd.io/v1beta3 ✔ receivers.notification.toolkit.fluxcd.io/v1 ✗ check failed

Git provider

GitHub

Container Registry provider

quay.io

Additional context

No response

Code of Conduct

stefanprodan commented 3 days ago

The ImagePolicy is missing the labels you've set in the ImageUpdateAutomation, so I'm very surprised it works when you run reconcile, it should never update the repo.