karmada-io / karmada

Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
Apache License 2.0
3.54k stars 742 forks source link
cloud-computing cloud-native containers k8s kubernetes multi-cluster multicloud



LICENSE Releases Slack CII Best Practices build Go Report Card codecov

Karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration

Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications. By speaking Kubernetes-native APIs and providing advanced scheduling capabilities, Karmada enables truly open, multi-cloud Kubernetes.

Karmada aims to provide turnkey automation for multi-cluster application management in multi-cloud and hybrid cloud scenarios, with key features such as centralized multi-cloud management, high availability, failure recovery, and traffic scheduling.


Karmada is a sandbox project of the Cloud Native Computing Foundation (CNCF).

Why Karmada:

Notice: this project is developed in continuation of Kubernetes Federation v1 and v2. Some basic concepts are inherited from these two versions.



The Karmada Control Plane consists of the following components:

ETCD stores the Karmada API objects, the API Server is the REST endpoint all other components talk to, and the Karmada Controller Manager performs operations based on the API objects you create through the API server.

The Karmada Controller Manager runs the various controllers, the controllers watch Karmada objects and then talk to the underlying clusters' API servers to create regular Kubernetes resources.

  1. Cluster Controller: attach Kubernetes clusters to Karmada for managing the lifecycle of the clusters by creating cluster objects.
  2. Policy Controller: the controller watches PropagationPolicy objects. When the PropagationPolicy object is added, it selects a group of resources matching the resourceSelector and creates ResourceBinding with each single resource object.
  3. Binding Controller: the controller watches ResourceBinding object and create Work object corresponding to each cluster with a single resource manifest.
  4. Execution Controller: the controller watches Work objects. When Work objects are created, it will distribute the resources to member clusters.


Resource template: Karmada uses Kubernetes Native API definition for federated resource template, to make it easy to integrate with existing tools that already adopt on Kubernetes

Propagation Policy: Karmada offers a standalone Propagation(placement) Policy API to define multi-cluster scheduling and spreading requirements.

Override Policy: Karmada provides standalone Override Policy API for specializing cluster relevant configuration automation. E.g.:

The following diagram shows how Karmada resources are involved when propagating resources to member clusters.


Quick Start

This guide will cover:


Install the Karmada control plane

1. Clone this repo to your machine:

git clone https://github.com/karmada-io/karmada

2. Change to the karmada directory:

cd karmada

3. Deploy and run Karmada control plane:

run the following script:


This script will do the following tasks for you:

If everything goes well, at the end of the script output, you will see similar messages as follows:

Local Karmada is running.

To start using your Karmada environment, run:
  export KUBECONFIG="$HOME/.kube/karmada.config"
Please use 'kubectl config use-context karmada-host/karmada-apiserver' to switch the host and control plane cluster.

To manage your member clusters, run:
  export KUBECONFIG="$HOME/.kube/members.config"
Please use 'kubectl config use-context member1/member2/member3' to switch to the different member cluster.

There are two contexts in Karmada:

The karmada-apiserver is the main kubeconfig to be used when interacting with the Karmada control plane, while karmada-host is only used for debugging Karmada installation with the host cluster. You can check all clusters at any time by running: kubectl config view. To switch cluster contexts, run kubectl config use-context [CONTEXT_NAME]



Propagate application

In the following steps, we are going to propagate a deployment by Karmada.

1. Create nginx deployment in Karmada.

First, create a deployment named nginx:

kubectl create -f samples/nginx/deployment.yaml

2. Create PropagationPolicy that will propagate nginx to member cluster

Then, we need to create a policy to propagate the deployment to our member cluster.

kubectl create -f samples/nginx/propagationpolicy.yaml

3. Check the deployment status from Karmada

You can check deployment status from Karmada, don't need to access member cluster:

$ kubectl get deployment
nginx   2/2     2            2           20s

Kubernetes compatibility

Kubernetes 1.16 Kubernetes 1.17 Kubernetes 1.18 Kubernetes 1.19 Kubernetes 1.20 Kubernetes 1.21 Kubernetes 1.22 Kubernetes 1.23 Kubernetes 1.24 Kubernetes 1.25 Kubernetes 1.26
Karmada v0.9
Karmada v0.10
Karmada v1.0
Karmada v1.1
Karmada v1.2
Karmada v1.3
Karmada v1.4
Karmada v1.5
Karmada v1.6
Karmada HEAD (master)



Regular Community Meeting:



If you have questions, feel free to reach out to us in the following ways:

Talks and References

KubeCon(EU 2021) Beyond federation: automating multi-cloud workloads with K8s native APIs
KubeCon(EU 2022) Sailing Multi Cloud Traffic Management With Karmada
KubeDay(Israel 2023) Simplifying Multi-cluster Kubernetes Management with Karmada

For blogs, please refer to website.


If you're interested in being a contributor and want to get involved in developing the Karmada code, please see CONTRIBUTING for details on submitting patches and the contribution workflow.


Karmada is under the Apache 2.0 license. See the LICENSE file for details.