SparebankenVest / public-helm-charts

Public Helm charts provided by Sparebanken Vest
https://charts.spvapi.no
28 stars 51 forks source link

Error Liveness probe failed and Readiness probe failed in chartversion akv2k8s-2.4.2 #114

Open Didjacome opened 1 year ago

Didjacome commented 1 year ago

Good evening everybody, I hope you are well

I have a problem installing the CHART akv2k8s-2.4.2 version, whenever it is installed the controller pod goes into the CrashLoopBackOff state because of its Liveness probe and Readiness probe settings and this interferes even in the controller scale

To be able to reproduce the error:

helm repo add spv-charts https://charts.spvapi.no helm repo update helm upgrade --namespace akv2k8s --install --set global.logLevel=debug --set global.logFormat=text my-akv2k8s-operators spv-charts/akv2k8s

I verified that this update is based on this commit 82f413c7f1d22ded69bcb1834184967ff77c3693

but the error is because the application either doesn't know the /healthz path, or it doesn't listen to port 9000

Error:

Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  57m                   default-scheduler  Successfully assigned akv2k8s/my-akv2k8s-operators-controller-XXXXXXX-dtwqv to aks-nodepoll-XXXXX-vmssXXXXXXXX
  Normal   Pulling    57m                   kubelet            Pulling image "spvest/azure-keyvault-controller:1.4.0"
  Normal   Pulled     56m                   kubelet            Successfully pulled image "spvest/azure-keyvault-controller:1.4.0" in 3.200184061s (33.273565694s including waiting)
  Normal   Killing    56m                   kubelet            Container controller failed liveness probe, will be restarted
  Normal   Created    56m (x2 over 56m)     kubelet            Created container controller
  Normal   Started    56m (x2 over 56m)     kubelet            Started container controller
  Normal   Pulled     56m                   kubelet            Container image "spvest/azure-keyvault-controller:1.4.0" already present on machine
  Warning  Unhealthy  55m (x5 over 56m)     kubelet            Liveness probe failed: Get "http://192.168.0.10:9000/healthz": dial tcp 192.168.0.10:9000: connect: connection refused
  Warning  Unhealthy  7m6s (x135 over 56m)  kubelet            Readiness probe failed: Get "http://192.168.0.10:9000/healthz": dial tcp 192.168.0.10:9000: connect: connection refused
jpwig commented 1 year ago

I'm having the same issue.

image

Aleksey-Kudryavtsev commented 1 year ago

I'm seeing the same issue Had to add

global:
  metrics:
    enabled: true

in values.yaml to fix it

Looks like it should not add the probes if this is disabled or the probes should be independent of metrics and enabled by default.