search

argoproj / argo-cd

Declarative Continuous Deployment for Kubernetes
https://argo-cd.readthedocs.io
Apache License 2.0
18k stars 5.48k forks source link

Latest stable release and Getting Started guide is using wrong image #20893

Closed DecseiD closed 18 hours ago

DecseiD commented 21 hours ago

Checklist:

Describe the bug

I have a single-server K8s cluster running on my Proxmox server at home. I followed the "Getting started" guide from the official page to deploy ArgoCD on my cluster.

The "argocd-dex-server" deployment's pod was stuck in ImagePullBackOff error due to not being able to pull the image. From the current "Getting started" command, we can see the following RAW output: ... image: ghcr.io/dexidp/dex:v2.41.1 ...

I have completely open internet access on my server and everything else was able to get pulled and started.

To Reproduce

Deploy a new instance of ArgoCD by following ther "Getting started" guide:

kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Expected behavior

All components should be coming up without any manual intervention.

Screenshots By the time of the screenshot, I replaced the image - details are below as last point)

image

Version

argocd: v2.13.1+af54ef8 BuildDate: 2024-11-20T15:58:16Z GitCommit: af54ef8db5adfa77a08d4d05b1318a2198084c22 GitTreeState: clean GoVersion: go1.23.1 Compiler: gc Platform: linux/amd64

Logs

kubectl describe pod -n argocd argocd-dex-server-84b879d87c-kxrg8

argocd Failed to pull image "ghcr.io/dexidp/dex:v2.41.1": failed to pull and unpack image "ghcr.io/dexidp/dex:v2.41.1": failed to resolve reference "ghcr.io/dexidp/dex:v2.41.1": failed to do request: Head "https://ghcr.io/v2/dexidp/dex/manifests/v2.41.1": dial tcp 140.82.121.34:443: connect: no route to

Workaround

I did manage to solve this by manually changing the deployment's image to "bitnami/dex:2.41.1-debian-12-r8" that I saw in an ArgoCD-related PR but I'm not sure if this was the appropriate step.

andrii-korotkov-verkada commented 20 hours ago

Hm, it seems to be there https://github.com/dexidp/dex/pkgs/container/dex/254808321?tag=v2.41.1

andrii-korotkov-verkada commented 19 hours ago

Can you debug your connectivity to ghcr.io, please? In particular, from the host in the cluster (tho it may be not the same).

DecseiD commented 18 hours ago

hello @andrii-korotkov-verkada

I was troubleshooting this on my and interestingly it is related to my networking. When I use static IP for my VM, I'm not able to access "ghcr.io" (No route to host) but when I reset to default (DHCP), I can get the image as expected:

This is from my Proxmox server directly:

root@ddecsei:~# docker pull ghcr.io/dexidp/dex:v2.41.1 v2.41.1: Pulling from dexidp/dex c6a83fedfae6: Pull complete c453ee6f343a: Pull complete 3366c9cf7837: Pull complete 85f1e9196f5c: Pull complete 14dc2d53164e: Pull complete 9fbb810783db: Pull complete 4903bb51dfe0: Pull complete 7c85bc8e8ed2: Pull complete 7e8f7aff942d: Pull complete d0ac2d9c6e69: Pull complete Digest: sha256:bc7cfce7c17f52864e2bb2a4dc1d2f86a41e3019f6d42e81d92a301fad0c8a1d Status: Downloaded newer image for ghcr.io/dexidp/dex:v2.41.1 ghcr.io/dexidp/dex:v2.41.1

This is from the Single server CP node:

ddecsei@controlplane-1:~$ sudo docker pull ghcr.io/dexidp/dex:v2.41.1 [sudo] password for ddecsei: v2.41.1: Pulling from dexidp/dex c6a83fedfae6: Pull complete c453ee6f343a: Pull complete 3366c9cf7837: Pull complete 85f1e9196f5c: Pull complete 14dc2d53164e: Pull complete 9fbb810783db: Pull complete 4903bb51dfe0: Pull complete 7c85bc8e8ed2: Pull complete 7e8f7aff942d: Pull complete d0ac2d9c6e69: Pull complete Digest: sha256:bc7cfce7c17f52864e2bb2a4dc1d2f86a41e3019f6d42e81d92a301fad0c8a1d Status: Downloaded newer image for ghcr.io/dexidp/dex:v2.41.1 ghcr.io/dexidp/dex:v2.41.1

It is still an interesting question though that why am I able to access everything else (like the image I used as a replacement) but the one at "ghcr.io"?

(This bug can be closed as the root-cause was my network setup but another one might be beneficial to find out what other network setup will not work with "ghcr.io")

Is there any specific reason to keep that one deployment image on ghcr.io instead of dockerhub for example?

andrii-korotkov-verkada commented 18 hours ago

Okay. You can also try mirroring the image internally and having an overlay configuring new location, if full network access is undesirable.