search

crossplane-contrib / provider-helm

Crossplane Helm Provider
Apache License 2.0
98 stars 63 forks source link

howto debugging reconciling loop #199

Open eumel8 opened 10 months ago

eumel8 commented 10 months ago

What happened?

Using crossplane provider-helm to deploy vcluster chart. The installation goes never done due non ready state:

NAME        CHART      VERSION   SYNCED   READY   STATE      REVISION   DESCRIPTION        AGE
vc1-8d6xs   vcluster   0.15.7    True     False   deployed   763        Upgrade complete   16m

The debug log shows not exactly what is changed or missing:

2023-09-12T19:28:10.548Z    DEBUG   provider-helm   Looks like there are no changes for [ClusterRoleBinding vc-vc1-vcluster-v-vc1] %!q(MISSING)
2023-09-12T19:28:10.559Z    DEBUG   provider-helm   Looks like there are no changes for [Role vc1-vcluster] %!q(MISSING)
2023-09-12T19:28:10.574Z    DEBUG   provider-helm   Looks like there are no changes for [RoleBinding vc1-vcluster] %!q(MISSING)
2023-09-12T19:28:10.584Z    DEBUG   provider-helm   Looks like there are no changes for [Service vc1-vcluster] %!q(MISSING)
2023-09-12T19:28:10.594Z    DEBUG   provider-helm   Looks like there are no changes for [Service vc1-vcluster-headless] %!q(MISSING)
2023-09-12T19:28:10.606Z    DEBUG   provider-helm   Patch [StatefulSet vc1-vcluster vc1] %!q(MISSING) in namespace %!s(MISSING)
2023-09-12T19:28:10.671Z    DEBUG   provider-helm   updating status for upgraded release for [vc1-vcluster]
2023-09-12T19:28:10.711Z    DEBUG   provider-helm   Successfully requested update of external resource  {"controller": "managed/release.helm.crossplane.io", "request": "/vc1-8d6xs", "uid": "e7e9f382-6cb7-4e3c-bf5c-a5e28793bc76", "version": "72974825", "external-name": "vc1-vcluster", "requeue-after": "2023-09-12T19:38:10.711Z"}
2023-09-12T19:28:10.712Z    DEBUG   events  Successfully requested update of external resource  {"type": "Normal", "object": {"kind":"Release","name":"vc1-8d6xs","uid":"e7e9f382-6cb7-4e3c-bf5c-a5e28793bc76","apiVersion":"helm.crossplane.io/v1beta1","resourceVersion":"72974825"}, "reason": "UpdatedExternalResource"}
2023-09-12T19:28:10.723Z    DEBUG   provider-helm   Reconciling {"controller": "managed/release.helm.crossplane.io", "request": "/vc1-8d6xs"}
2023-09-12T19:28:10.723Z    DEBUG   provider-helm   Connecting  {"request": "vc1-8d6xs"}
2023-09-12T19:28:10.739Z    DEBUG   provider-helm   Observing   {"request": "vc1-8d6xs"}
2023-09-12T19:28:10.816Z    DEBUG   provider-helm   Updating    {"request": "vc1-8d6xs"}
2023-09-12T19:28:10.821Z    DEBUG   provider-helm   preparing upgrade for [vc1-vcluster]
2023-09-12T19:28:10.893Z    DEBUG   provider-helm   resetting values to the chart's original version%!(EXTRA []interface {}=[])

How can we reproduce it?

related release:

apiVersion: helm.crossplane.io/v1beta1
kind: Release
metadata:
  annotations:
    crossplane.io/composition-resource-name: vcluster-helm-release
    crossplane.io/external-create-pending: "2023-09-12T19:16:30Z"
    crossplane.io/external-create-succeeded: "2023-09-12T19:16:30Z"
    crossplane.io/external-name: vc1-vcluster
  creationTimestamp: "2023-09-12T19:16:30Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generateName: vc1-
  generation: 3
  labels:
    crossplane.io/claim-name: ""
    crossplane.io/claim-namespace: ""
    crossplane.io/composite: vc1
  name: vc1-8d6xs
  ownerReferences:
  - apiVersion: caas.telekom.de/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Vcluster
    name: vc1
    uid: 035ffe98-7a52-49d3-abe6-1b8b557f96ef
  resourceVersion: "72983650"
  uid: e7e9f382-6cb7-4e3c-bf5c-a5e28793bc76
spec:
  deletionPolicy: Delete
  forProvider:
    chart:
      name: vcluster
      repository: oci://mtr.devops.telekom.de/caas/charts
      url: oci://mtr.devops.telekom.de/caas/charts/vcluster:0.15.2
      version: 0.15.7
    namespace: vc1
    values:
      coredns:
        image: mtr.devops.telekom.de/rancher/mirrored-coredns-coredns:1.10.1
      defaultImageRegistry: mtr.devops.telekom.de
      storage:
        className: longhorn
      sync:
        hoststorageclasses:
          enabled: true
        ingresses:
          enabled: true
      syncer:
        image: /caas/vcluster:0.15.2
      vcluster:
        extraArgs:
        - --system-default-registry=mtr.devops.telekom.de
        image: /rancher/k3s:v1.26.0-k3s2-amd64
        resources:
          limits:
            cpu: 200m
            memory: 1Gi
          requests:
            cpu: 50m
            memory: 16Mi
  providerConfigRef:
    name: provider-helm
  rollbackLimit: 3
status:
  atProvider:
    releaseDescription: Upgrade complete
    revision: 999
    state: deployed
  conditions:
  - lastTransitionTime: "2023-09-12T19:16:32Z"
    reason: Unavailable
    status: "False"
    type: Ready
  - lastTransitionTime: "2023-09-12T19:16:31Z"
    reason: ReconcileSuccess
    status: "True"
    type: Synced

What environment did it happen in?

Crossplane version: v1.13.2 Crossplane-Helm-Version: v0.15.2 Kubernetes: v1.24.16

eumel8 commented 10 months ago

correction: this occurs only in Helm Chart Vcluster Version 0.15.7 because the state is Unavailable

Downgrade 0.15.2 works, but hasn't the security features what we need. Still difficult to find out the root cause