search

argoproj / argo-cd

Declarative Continuous Deployment for Kubernetes
https://argo-cd.readthedocs.io
Apache License 2.0
17.41k stars 5.29k forks source link

CreateContainerConfigError on multiple pods after install #18993

Open dove-young opened 2 months ago

dove-young commented 2 months ago

Checklist:

Describe the bug

I got multiple pods in CreateContainerConfigError status after install via instruction here https://argo-cd.readthedocs.io/en/stable/getting_started/

[root@api.] # oc get po -n argocd
NAME                                               READY   STATUS                       RESTARTS      AGE
argocd-application-controller-0                    0/1     CreateContainerConfigError   0             4m40s
argocd-applicationset-controller-65bb5ff89-6w4rg   1/1     Running                      0             4m41s
argocd-dex-server-6f898cbd9-mc4qz                  0/1     CrashLoopBackOff             5 (99s ago)   4m41s
argocd-notifications-controller-64bc7c9f7-gtw9k    1/1     Running                      0             4m41s
argocd-repo-server-74d5f58dc5-cwr7b                0/1     CreateContainerConfigError   0             4m40s
argocd-server-5b86767ddb-tt9zj                     0/1     CreateContainerConfigError   0             4m40s

I described pod argocd-repo-server it said Error: secret "argocd-redis" not found

  Normal   Pulling         2m3s (x7 over 3m13s)  kubelet            Pulling image "quay.io/argoproj/argocd:v2.11.3"
  Warning  Failed          2m2s (x7 over 3m12s)  kubelet            Error: secret "argocd-redis" not found
  Normal   Pulled          2m2s                  kubelet            Successfully pulled image "quay.io/argoproj/argocd:v2.11.3" in 903ms (903ms including waiting)

I described pod argocd-dex-server, it said Back-off restarting

  Normal   Pulling         6m40s (x4 over 7m19s)   kubelet            Pulling image "ghcr.io/dexidp/dex:v2.38.0"
  Normal   Started         6m39s (x4 over 7m18s)   kubelet            Started container dex
  Normal   Created         6m39s (x4 over 7m18s)   kubelet            Created container dex
  Normal   Pulled          6m39s                   kubelet            Successfully pulled image "ghcr.io/dexidp/dex:v2.38.0" in 725ms (725ms including waiting)
  Warning  BackOff         2m21s (x25 over 7m15s)  kubelet            Back-off restarting failed container dex in pod argocd-dex-server-6f898cbd9-mc4qz_argocd(457b4b9f-90b5-47d8-ac63-ddb5766b9a50)

Then I checked log from it, and it said

time="2024-07-09T02:49:23Z" level=info msg="Generating self-signed TLS certificate for this session"
time="2024-07-09T02:49:23Z" level=info msg="Starting configmap/secret informers"
time="2024-07-09T02:49:23Z" level=info msg="Configmap/secret informer synced"
time="2024-07-09T02:49:23Z" level=fatal msg="server.secretkey is missing"

I described pod argocd-server, it said Error: secret "argocd-redis" not found

  Warning  Failed          7m39s (x8 over 9m3s)   kubelet            Error: secret "argocd-redis" not found
  Normal   Pulled          7m39s                  kubelet            Successfully pulled image "quay.io/argoproj/argocd:v2.11.3" in 966ms (966ms including waiting)
  Normal   Pulling         4m3s (x23 over 9m10s)  kubelet            Pulling image "quay.io/argoproj/argocd:v2.11.3"

I checked log from it, it said

oc logs argocd-server-5b86767ddb-tt9zj
Error from server (BadRequest): container "argocd-server" in pod "argocd-server-5b86767ddb-tt9zj" is waiting to start: CreateContainerConfigError

To Reproduce

on OCP 4.15.18, install argocd on instruction right here https://argo-cd.readthedocs.io/en/stable/getting_started/

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Expected behavior

Screenshots

Version

Paste the output from `argocd version` here.

Logs

Paste any relevant application logs here.
christianh814 commented 2 months ago

This seems the same as #18696

Tagging @sabre1041 @ishitasequeira and @jannfis for completeness. I think the prefered way to install Argo CD is to use the Argo CD Operator

DevOpsJon commented 1 month ago

It seems like you need to create the secrets first. What worked for me is:

 echo -n 'SillytestPass' | base64
 kubectl create secret generic argocd-redis -n argocd --from-literal=auth=<output of above>
DesislavaNPetrova commented 3 weeks ago

Same issue or me here (v2.11.3, v2.12.0) during upgrade. The workaround with the secret fix "CreateContainerConfigError", but than got( register in v2.7.18 -->) "Error: unknown command "/usr/local/bin/argocd-server" for "argocd-server"" - https://github.com/argoproj/argo-cd/issues/17090.

Gen1usCat commented 3 weeks ago

Try to check argocd-redis deployment replicaset. I had the event 0 nodes are available: didn't match anti-affinity rules.
Look into install.yaml manifest for "redis-initial-password" and find that argocd-redis secret should be creating by init container from argocd-redis deployment pod.
I use the ha install, in my case the deployment name is argocd-redis-ha-haproxy. My test cluster has only 3 worker nodes and all 3 nodes had pods of argocd-redis-ha-haproxy.
Workaround for me: delete one by one old pods and wait for running new pods from new replicaset.