search

replicatedhq / outdated

Kubectl plugin to find and report outdated images running in a Kubernetes cluster
https://outdated.sh
Apache License 2.0
424 stars 23 forks source link

Extremely slow fetch speed #25

Open psarossy opened 3 years ago

psarossy commented 3 years ago

In the demo video it seems that this tool supposed to be able to fetch the latest/behind data in seconds per image. On my cluster, it takes minutes per image, and since I have like 50 different images, after like 3-4 of them it also just times out and says "Unable to get image data" and exist...

I'm using a mix of docker.io, quay.io, ghcr.io and gcr.io

Any hints on whether I'm doing something incorrectly or if this is normal?

marccampbell commented 3 years ago

Hi @psarossy. That doesn't sound normal or expected.

Could you provide an example of one of the specific images (if public)? To help debug this, it would be really useful if you are able to share one slow image name/tag that you are running.

Outdated may have to make a several requests to the registry to find the number of tags "out of date". I can think of a few reasons this might get slow, but I'd be happy to debug the network requests from one of your public images, in order to start narrowing it down.

psarossy commented 3 years ago

Sure, here's the log from a fresh run. They are pretty common/widely used images...

$ time kubectl outdated                                                                                                                                                         [17:40:23]

Image                                          Current                                                  Latest                                                  Behind
index.docker.io/adguard/adguardhome            latest                                                   latest                                                  2
index.docker.io/linuxserver/nzbget             latest                                                   latest                                                  6
index.docker.io/linuxserver/radarr             latest                                                   latest                                                  0
index.docker.io/linuxserver/sonarr             preview                                                  Unable to get image data
index.docker.io/linuxserver/transmission       latest                                                   Unable to get image data
index.docker.io/bitwardenrs/server             latest                                                   Unable to get image data
index.docker.io/ottomated/crewlink-server      latest                                                   Unable to get image data
index.docker.io/saspus/duplicacy-web           latest                                                   Unable to get image data
index.docker.io/f0rc3/barcodebuddy-docker      latest                                                   Unable to get image data
ghcr.io/linuxserver/grocy                      latest                                                   Unable to get image data
index.docker.io/homeassistant/home-assistant   stable                                                   Unable to get image data
index.docker.io/homeassistant/home-assistant   stable                                                   Unable to get image data
index.docker.io/library/mariadb                10.5.8                                                   Unable to get image data
index.docker.io/thomseddon/traefik-forward-auth2                                                        Unable to get image data
index.docker.io/marthoc/homeseer               latest                                                   Unable to get image data
index.docker.io/calico/kube-controllers        v3.16.5                                                  Unable to get image data
index.docker.io/calico/cni                     v3.16.5                                                  Unable to get image data
index.docker.io/calico/pod2daemon-flexvol      v3.16.5                                                  Unable to get image data
index.docker.io/calico/node                    v3.16.5                                                  Unable to get image data
k8s.gcr.io/library/coredns                     1.7.0                                                    Unable to get image data
k8s.gcr.io/library/etcd                        3.4.13-0                                                 Unable to get image data
index.docker.io/library/haproxy                2.1.4                                                    Unable to get image data
index.docker.io/herbrandson/k8dash             latest                                                   Unable to get image data
index.docker.io/osixia/keepalived              1.3.5-1                                                  Unable to get image data
k8s.gcr.io/library/kube-apiserver              v1.19.6                                                  Unable to get image data
k8s.gcr.io/library/kube-controller-manager     v1.19.6                                                  Unable to get image data
index.docker.io/nfvpe/multus                   stable                                                   Unable to get image data
k8s.gcr.io/library/kube-proxy                  v1.19.6                                                  Unable to get image data
k8s.gcr.io/library/kube-scheduler              v1.19.6                                                  Unable to get image data
k8s.gcr.io/metrics-server/metrics-server       v0.3.7                                                   Unable to get image data
index.docker.io/nvidia/k8s-device-plugin       v0.7.1                                                   Unable to get image data
index.docker.io/library/traefik                2.2.1                                                    Unable to get image data
index.docker.io/containous/whoami              latest                                                   Unable to get image data
index.docker.io/kubernetesui/metrics-scraper   v1.0.4                                                   Unable to get image data
index.docker.io/kubernetesui/dashboard         v2.0.0                                                   Unable to get image data
index.docker.io/kubevirt/virt-api              sha256:4d778f63d2f5ecb61d4f6fe8b5ead010836d46d7...       Unable to get image data
index.docker.io/library/sha256                 b77437d649aacb6f731475d9545ab83d6b083653f8c94a3...       Unable to get image data
index.docker.io/library/sha256                 431713e785215224d48ca265183d4afa098f13bde25f035...       Unable to get image data
index.docker.io/library/sha256                 8e2d63c20706fbeb13ca24c27ed3d07b197ed54d4024726...       Unable to get image data
index.docker.io/metallb/controller             v0.9.3                                                   Unable to get image data
index.docker.io/metallb/speaker                v0.9.3                                                   Unable to get image data
index.docker.io/psarossy/docker-bloomsky       latest                                                   Unable to get image data
index.docker.io/psarossy/btmon-influx          latest                                                   Unable to get image data
index.docker.io/library/chronograf             1.8                                                      Unable to get image data
index.docker.io/grafana/grafana                7.1.5                                                    Unable to get image data
index.docker.io/library/influxdb               1.8                                                      Unable to get image data
index.docker.io/library/telegraf               1.5.2                                                    Unable to get image data
index.docker.io/golift/unifi-poller            stable                                                   Unable to get image data
index.docker.io/vernemq/vernemq                1.10.3-alpine                                            Unable to get image data
ghcr.io/linuxserver/nextcloud                  latest                                                   Unable to get image data
index.docker.io/nodered/node-red               latest                                                   Unable to get image data
index.docker.io/octoprint/octoprint            1.4.2                                                    Unable to get image data
index.docker.io/owntracks/recorder             latest                                                   Unable to get image data
index.docker.io/linuxserver/papermerge         latest                                                   Unable to get image data
index.docker.io/photoprism/photoprism          latest                                                   Unable to get image data
index.docker.io/linuxserver/ombi               latest                                                   Unable to get image data
index.docker.io/linuxserver/plex               latest                                                   Unable to get image data
index.docker.io/linuxserver/tautulli           latest                                                   Unable to get image data
index.docker.io/linuxserver/resilio-sync       latest                                                   Unable to get image data
quay.io/cephcsi/cephcsi                        v2.1.2                                                   Unable to get image data
quay.io/k8scsi/csi-node-driver-registrar       v1.2.0                                                   Unable to get image data
quay.io/k8scsi/csi-attacher                    v2.1.0                                                   Unable to get image data
quay.io/k8scsi/csi-provisioner                 v1.4.0                                                   Unable to get image data
quay.io/k8scsi/csi-resizer                     v0.4.0                                                   Unable to get image data
quay.io/k8scsi/csi-snapshotter                 v1.2.2                                                   Unable to get image data
index.docker.io/ceph/ceph                      v14.2.9                                                  Unable to get image data
index.docker.io/ceph/ceph                      v14.2.9                                                  Unable to get image data
index.docker.io/ceph/ceph                      v14.2.9                                                  Unable to get image data
index.docker.io/rook/ceph                      v1.3.11                                                  Unable to get image data
index.docker.io/rook/ceph                      v1.3.1                                                   Unable to get image data
index.docker.io/rook/ceph                      master                                                   Unable to get image data
index.docker.io/dperson/samba                  latest                                                   Unable to get image data
index.docker.io/library/mysql                  5                                                        Unable to get image data
index.docker.io/library/busybox                latest                                                   Unable to get image data
index.docker.io/linuxserver/snipe-it           latest                                                   Unable to get image data
ghcr.io/linuxserver/code-server                latest                                                   Unable to get image data

kubectl outdated  12.33s user 3.29s system 5% cpu 5:00.98 total
marccampbell commented 3 years ago

I've reproduced this, and it seems that we are pulling each tag separately to get the dates from the manifest of the tag. For non-semver tags, we do this to figure out the sort order.

Maybe we can change the product a little so instead of showing the number of releases behind, we show that you are behind and the latest release. Once we find 3 or 5 releases between the 2, we can show the output as red, and say something like "5+ releases behind" instead of counting each

FalconerTC commented 1 year ago

+1 still seeing this