senthilrch / kube-fledged

A kubernetes operator for creating and managing a cache of container images directly on the cluster worker nodes, so application pods start almost instantly
Apache License 2.0
1.24k stars 118 forks source link
add-on cache container delete go golang image k8s kubernetes operator pull registry repository tool

Build Status Coverage Status Go Report Card GitHub release (latest by date) License

kube-fledged is a kubernetes operator for creating and managing a cache of container images directly on the worker nodes of a kubernetes cluster. It allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pulled). As a result, application pods start almost instantly, since the images need not be pulled from the registry.

kube-fledged provides CRUD APIs to manage the lifecycle of the image cache, and supports several configurable parameters to customize the functioning as per one's needs.

Table of contents

Use cases

Prerequisites

Quick Install using YAML manifests

These instructions install kube-fledged to a separate namespace called "kube-fledged", using YAML manifests and pre-built images in Docker Hub.

Quick Install using Helm chart

Quick Install using Helm operator

These instructions install kube-fledged to a separate namespace called "kube-fledged", using Helm operator and pre-built images in Docker Hub.

Helm chart parameters

Parameters of the helm chart are documented here

Build and Deploy

These instructions will help you build kube-fledged from source and deploy it to a separate namespace called "kube-fledged". If you need to deploy it to a different namespace, edit the namespace field of the manifests in "kube-fledged/deploy" accordingly.

Build

Deploy

Note:- You need to have 'cluster-admin' privileges to deploy

How to use

kube-fledged provides APIs to perform CRUD operations on image cache. These APIs can be consumed via kubectl or curl

Create image cache

Refer to sample image cache manifest in "deploy/kubefledged-imagecache.yaml". Edit it as per your needs before creating image cache. If images are in private repositories requiring credentials to pull, add "imagePullSecrets" to the end.

  imagePullSecrets:
  - name: myregistrykey

Create the image cache using kubectl. Verify successful creation

$ kubectl create -f deploy/kubefledged-imagecache.yaml
$ kubectl get imagecaches -n kube-fledged

View the status of image cache

Use following command to view the status of image cache in "json" format.

$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Add/remove images in image cache

Use kubectl edit command to add/remove images in image cache. The edit command opens the manifest in an editor. Edit your changes, save and exit.

$ kubectl edit imagecaches imagecache1 -n kube-fledged
$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Refresh image cache

kube-fledged supports both automatic and on-demand refresh of image cache. Auto refresh is enabled using the flag --image-cache-refresh-frequency:. To request for an on-demand refresh, run the following command:-

$ kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=

Delete image cache

Before you could delete the image cache, you need to purge the images in the cache using the following command. This will remove all cached images from the worker nodes.

$ kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/purge-imagecache=

View the status of purging the image cache. If any failures, such images should be removed manually or you could decide to leave the images in the worker nodes.

$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Finally delete the image cache using following command.

$ kubectl delete imagecaches imagecache1 -n kube-fledged

Remove kube-fledged

Run the following command to remove kube-fledged from the cluster.

$ make remove-kubefledged (if you deployed using YAML manifests)
$ helm delete kube-fledged -n ${KUBEFLEDGED_NAMESPACE} (if you deployed using Helm chart)
$ make remove-kubefledged-and-operator (if you deployed using Helm Operator)

How it works

Kubernetes allows developers to extend the kubernetes api via Custom Resources. kube-fledged defines a custom resource of kind “ImageCache” and implements a custom controller (named kubefledged-controller). kubefledged-controller does the heavy-lifting for managing image cache. Users can use kubectl commands for creation and deletion of ImageCache resources.

kubefledged-controller has a built-in image manager routine that is responsible for pulling and deleting images. Images are pulled or deleted using kubernetes jobs. If enabled, image cache is refreshed periodically by the refresh worker. kubefledged-controller updates the status of image pulls, refreshes and image deletions in the status field of ImageCache resource.

For more detailed description, go through kube-fledged's design proposal.

Configuration Flags for Kubefledged Controller

--cri-socket-path: path to the cri socket on the node e.g. /run/containerd/containerd.sock (default: /var/run/docker.sock, /run/containerd/containerd.sock, /var/run/crio/crio.sock)

--image-cache-refresh-frequency: The image cache is refreshed periodically to ensure the cache is up to date. Setting this flag to "0s" will disable refresh. default "15m"

--image-delete-job-host-network: Whether the pod for the image delete job should be run with 'HostNetwork: true'. Default value: false.

--image-pull-deadline-duration: Maximum duration allowed for pulling an image. After this duration, image pull is considered to have failed. default "5m"

--image-pull-policy: Image pull policy for pulling images into and refreshing the cache. Possible values are 'IfNotPresent' and 'Always'. Default value is 'IfNotPresent'. Image with no or ":latest" tag are always pulled.

--job-priority-class-name: priorityClassName of jobs created by kubefledged-controller.

--job-retention-policy: Determines if the jobs created by kubefledged-controller would be deleted or retained (for debugging) after it finishes. Possible values are 'delete' and 'retain'. default value is 'delete'.

--service-account-name: serviceAccountName used in Jobs created for pulling or deleting images. Optional flag. If not specified the default service account of the namespace is used

--stderrthreshold: Log level. set the value of this flag to INFO

Supported Container Runtimes

Supported Platforms

Built With

Blogs and Presentations

Shout-outs

Please read CONTRIBUTING.md for details on the process for submitting pull requests.

Contributors

Code of Conduct

Please read CODE_OF_CONDUCT.md for details on our code of conduct, and how to report violations.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.