search

jenkins-x / jx

Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton
https://jenkins-x.io/
Apache License 2.0
4.56k stars 786 forks source link

`jx sync` attempts to pull an invalid ksync image #6262

Closed robertnowell closed 4 years ago

robertnowell commented 4 years ago

Summary

jx sync attempts to pull an invalid ksync image

Have been working through @vfarcic's (excellent) DevOps 2.6 toolkit, and noticed an issue running jx sync: command was unable to result in successfully initialized pods.

$ jx sync --daemon
Downloading https://github.com/vapor-ware/ksync/releases/download/0.4.1/ksync_darwin_amd64 to /Users/robertnowell/.jx/bin/ksync.tmp...
Downloaded /Users/robertnowell/.jx/bin/ksync.tmp
Initialising ksync
==== Local Environment ====
Fetching extra binaries                     ✓

==== Preflight checks ====
Cluster Config                              ✓
Cluster Connection                          ✓
Cluster Version                             ✓
Cluster Permissions                         ✓

==== Cluster Environment ====
Adding ksync to the cluster                 ✓
Waiting for pods to be healthy              ✘
↳       Timed out waiting for the ksync daemonset to be healthy.

To debug, you can:
- Run 'kubectl --namespace=kube-system --context= get pods -lapp=ksync' to look at what's going on.
- Run 'ksync doctor' to do an in-depth check of your system and the cluster.
FATA[0100]                                              
ERROR: Error: Command failed  ksync init --upgrade --image vaporio/ksync:0.4.1
$ go-demo-6 % kubectl get pods --all-namespaces                                                                                                        
...
kube-system            ksync-8cwf7                                                0/2     ImagePullBackOff   0          12m
kube-system            ksync-fdvck                                                0/2     ImagePullBackOff   0          12m
kube-system            ksync-ppfd8                                                0/2     ImagePullBackOff   0          12m
$ go-demo-6 % kubectl -n kube-system describe pod ksync-8cwf7                                                                                          
...
Controlled By:  DaemonSet/ksync
Containers:
  ksync:
    Container ID:  
    Image:         vaporio/ksync:0.4.1
...
Events:
  Type     Reason     Age                   From                                              Message
  ----     ------     ----                  ----                                              -------
  Normal   Scheduled  22m                   default-scheduler                                 Successfully assigned kube-system/ksync-8cwf7 to gke-jx-rocks-default-pool-baa7e80e-tmfb
  Normal   BackOff    22m (x5 over 22m)     kubelet, gke-jx-rocks-default-pool-baa7e80e-tmfb  Back-off pulling image "vaporio/ksync:0.4.1"
  Warning  Failed     22m (x5 over 22m)     kubelet, gke-jx-rocks-default-pool-baa7e80e-tmfb  Error: ImagePullBackOff
  Normal   Pulling    22m (x3 over 22m)     kubelet, gke-jx-rocks-default-pool-baa7e80e-tmfb  pulling image "vaporio/ksync:0.4.1"
  Warning  Failed     22m (x3 over 22m)     kubelet, gke-jx-rocks-default-pool-baa7e80e-tmfb  Failed to pull image "vaporio/ksync:0.4.1": rpc error: code = Unknown desc = Error response from daemon: manifest for vaporio/ksync:0.4.1 not found

Appears that the image name is not quite right

The attempt to pull image vaporio/ksync:<version>seems expected from https://github.com/jenkins-x/jx/blob/master/pkg/cmd/sync/sync.go#L109

ksync images appear to exist at "ksync/ksync/":

$ go-demo-6 % docker pull ksync/ksync:git-a5628b9                                                                                                                                                                                                                                                          (master)go-demo-6
git-a5628b9: Pulling from ksync/ksync
89d9c30c1d48: Pull complete 
4c6a69e598b9: Pull complete 
89673ea5653c: Pull complete 
97b28c888ef9: Pull complete 
Digest: sha256:fcb07b0634571432d76736b695c676365907b4215d2f0edd179a6ef2df2fa486
Status: Downloaded newer image for ksync/ksync:git-a5628b9
docker.io/ksync/ksync:git-a5628b9
$ go-demo-6 % docker pull vaporio/ksync:0.4.1                                                                                                                                                                                                                                                              (master)go-demo-6
Error response from daemon: manifest for vaporio/ksync:0.4.1 not found: manifest unknown: manifest unknown
?1 go-demo-6 % docker pull ksync/ksync:0.4.1                                                                                                                                                                                                                                                               (master)go-demo-6
Error response from daemon: manifest for ksync/ksync:0.4.1 not found: manifest unknown: manifest unknown

Seems ksync may have changed their image names recently?

Jx version

The output of jx version is:

NAME               VERSION
jx                 2.0.1011
jenkins x platform 2.0.1642
Kubernetes cluster v1.13.11-gke.14
kubectl            v1.16.3
helm client        Client: v2.16.1+gbbdfe5e
git                2.20.1 (Apple Git-117)
Operating System   Mac OS X 10.14.6 build 18G95

Jenkins type

Kubernetes cluster

GKE, created through jx create cluster

anmiralles commented 4 years ago

+1

same issue here. Any updates on this to solve the problem?

Thanks ;)

restalion commented 4 years ago

We are also stuck at this point 😞

ankitm123 commented 4 years ago

The issue is that jx tries to use the git tag to pull the docker image, but the images are tagged with their git commit for ksync. So, for the latest version it should be:

docker pull ksync/ksync:git-b2ca492

This is the related issue: https://github.com/ksync/ksync/issues/323 I will make a PR with the fix.