senthilrch / kube-fledged

A kubernetes operator for creating and managing a cache of container images directly on the cluster worker nodes, so application pods start almost instantly
Apache License 2.0
1.26k stars 119 forks source link

Updating the list of "ImageCache" directly #159

Closed gantvi01 closed 2 years ago

gantvi01 commented 2 years ago

Hi @senthilrch

I can set up the Kube-fledged on my aks cluster, and we have about 15 images, and I can cache on respective nodes using nodeselector, Our images are in Jfrog, and they get updated when our devs make changes. Is there a solution or pattern you are aware of to update the list of images dynamically to the "imagecache"

Thank you for the detailed documentation.

gantvi01 commented 2 years ago

We are able to resolve this issue by using helm and linking that helm chart to argocd

Example template for each application

apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:
  # Name of the image cache. A cluster can have multiple image cache objects
  name: imagecache-{{ include "app.fullname" . }}
  # The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preference
  labels:
    app: kubefledged
    kubefledged: imagecache
spec:
  # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).
  cacheSpec:
  # Specifies a list of images (redis:6.2.5 and mariadb:10.5.11) with a node selector, hence these images will be cached only on the nodes selected by the node selector
  - images:
    - {{ include "image.finalname" . }}
    nodeSelector:
       agentpool: {{ .Values.image.cacheNodeSelector }}       
  # Specifies a list of image pull secrets to pull images from private repositories into the cache
  imagePullSecrets:
  - name: {{ .Values.image.pullSecretName }}

Example using over a range of images

{{- range $selector := .Values.nodeSelectors}}
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:
  # Name of the image cache. A cluster can have multiple image cache objects
  name: imagecache-bootstrap-{{ $selector }}
  # The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preference
  labels:
    app: kubefledged
    kubefledged: imagecache
spec:
  # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).
  cacheSpec:
  # Specifies a list of images (redis:6.2.5 and mariadb:10.5.11) with a node selector, hence these images will be cached only on the nodes selected by the node selector
  - images:
  {{- range $image := $.Values.images }}
    - {{ $image.name | quote }}
  {{- end }}
    nodeSelector:
       agentpool: {{ $selector }}       
  # Specifies a list of image pull secrets to pull images from private repositories into the cache
  imagePullSecrets:
  - name: regcred
---
{{- end }}
senthilrch commented 2 years ago

Brilliant!