Kamaji

🤔 What is Kamaji?

Kamaji is a Kubernetes Control Plane Manager leveraging on the concept of Hosted Control Plane.

Kamaji's approach is based on running the Kubernetes Control Plane components in Pods instead of dedicated machines. This allows operating Kubernetes clusters at scale, with a fraction of the operational burden. Thanks to this approach, running multiple Control Planes can be cheaper and easier to deploy and operate.

Kamaji is like a fleet of Site Reliability Engineers with expertise codified into its logic, working 24/7 to keep up and running your Control Planes.

📖 How it works

Kamaji is extending the Kubernetes API capabilities thanks to Custom Resource Definitions.

By installing Kamaji, two pairs of new APIs will be available:

• TenantControlPlane, the instance definition of your desired Kubernetes Control Plane
• Datastore, the backing store used by one (or more) TenantControlPlane

The TenantControlPlane (short-named as tcp) objects are Namespace-scoped and allows configuring every aspect of your desired Control Plane. Besides the Kubernetes configuration values, you can specify the Pod options such as limit, request, tolerations, node selector, etc., as well as how these should be exposed (e.g.: using a ClusterIP, a LoadBalancer, or a NodePort).

The TenantControlPlane is the stateless definition of the Control Plane allowing to set up the required components for a full-fledged Kubernetest cluster. The state is managed by the Datastore API, a cluster-scoped resource which can hold the data of one or more Kubernetes clusters.

For further information about the API specifications and all the available options, refer to the official API reference.

⭐️ Main features

• Fast provisioning time: depending on the infrastructure, Tenant Control Planes are up and ready to serve traffic in 16 seconds.
• Streamlined update: the rollout to a new Kubernetes version for a given Tenant Control Plane takes just 10 seconds, with a Blue/Green deployment to avoid serving mixed Kubernetes versions.
• Resource optimization: thanks to the Datastore decoupling, there's no need of odd number instances (e.g.: RAFT consensus) by allowing to save up to 60% of HW resources.
• Scale from zero to the moon: scale down the instance when there's no usage, or automatically scale to support the traffic spikes reusing the Kubernetes patterns.
• Declarative approach, constant reconciliation: thanks to the Operator pattern, drift detection happens in real-time, maintaining the desired state.
• Automated certificates management: Kamaji leverages on kubeadm and the certificates are automatically created and rotated for you.
• Managing core addons: Kamaji allows configuring automatically kube-proxy, CoreDNS, and konnectivity, with automatic remediation in case of user errors (e.g.: deleting the CoreDNS deployment).
• Auto Healing: the TenantControlPlane objects in the management cluster are tracked by Kamaji, in case of deletion of those, everything is created in an idempotent way.
• Datastore multi-tenancy: optionally, Kamaji allows running multiple Control Planes on the same Datastore instance leveraging on the multi-tenancy of each driver, decreasing operations and optimizing costs.
• Overcoming etcd limitations: optionally, Kamaji allows using a different Datastore thanks to kine by supporting MySQL, PostgreSQL, or NATS as an alternative.
• Simplifying mixed-networks setup: thanks to Konnectivity, the Tenant Control Plane is connected to the worker nodes hosted in a different network, overcoming the no-NAT availability when dealing with nodes with a non routable IP address (e.g.: worker nodes in a different infrastructure).

🚀 Use cases

• Creating a private Managed Kubernetes Service
• Building a Platform as a Service
• Overcoming public Managed Kubernetes Services such as EKS
• Hybrid infrastructures: host the Control Plane on the Cloud and worker nodes on prem or vice-versa, according to your needs.
• Kubernetes at the edge: take full advantage of the Kubernetes API Server as a service paradigm.
• Kubernetes Control Plane as a Service: centrally manage multiple Kubernetes clusters from a single management point (Multi-Cluster management).
• High-density Control Plane: place multiple control planes on the same infrastructure, instead of having dedicated machines for each control plane.
• Strong Multi-tenancy: leave users to access the control plane with admin permissions while keeping them isolated at the infrastructure level.
• Kubernetes Inception: use Kubernetes to manage Kubernetes with automation, high-availability, fault tolerance, and autoscaling out of the box.
• Bring Your Own Device: keep the control plane isolated from data plane. Worker nodes can join and run consistently from everywhere: cloud, edge, and data-center.
• Full CNCF compliant: all clusters are built with upstream Kubernetes binaries, resulting in full CNCF compliant Kubernetes clusters.

🤔 You'd like to do the same but don't know how? 💡 CLASTIX can help you with your needs!

🧑‍💻‍ Production grade

Kamaji is empowering several businesses, and it counts public adopters. Check out the adopters file to learn more.

🤗 If you're using Kamaji, share your love by opening a PR!

🍦 Vanilla Kubernetes clusters

Kamaji is not yet-another-Kubernetes distribution: you have full freedom on the technology stack to provide to end users. Kamaji is a perfect fit for Platform Engineering, hiding the complexity of the Control Plane management to developers and DevOps engineers.

The provided Kubernetes Control Planes are CNCF compliant clusters.

🐢 Cluster API support

Kamaji is not a Cluster API replacement, rather, it plays very well with it.

Since Kamaji is just focusing on the Control Plane a Kamaji's Cluster API Control Plane provider has been developed.

🛣️ Roadmap

• [x] Dynamic address on Load Balancer
• [x] Zero Downtime Tenant Control Plane upgrade
• [x] Join worker nodes from anywhere thanks to Konnectivity
• [x] Alternative datastore MySQL, PostgreSQL, NATS
• [x] Pool of multiple datastores
• [x] Seamless migration between datastores
• [ ] Automatic assignment to a datastore
• [ ] Autoscaling of Tenant Control Plane
• [x] Provisioning through Cluster APIs
• [ ] Terraform provider
• [ ] Custom Prometheus metrics

🎥 Multimedia

• Playlist ▶️ Tutorials and How-Tos by Dario Tranchitella, CLASTIX
• YouTube ▶️ Metal³ provisioning with Kamaji Hosted Control Planes by Huy Mai, Ericsson
• YouTube ▶️ Hands-on introduction to Kamaji
• YouTube ▶️ Scaling Kubernetes up to 1,000 Control Planes
• YouTube ▶️ Equinix, Kamaji, and Cluster API
• YouTube ▶️ Rancher & Kamaji: solving multitenancy challenges in the Kubernetes world
• YouTube ▶️ Enabling Self-Service Kubernetes clusters with Kamaji and Paralus
• YouTube ▶️ Hosted Control Plane on Kubernetes (HPC) with Kamaji and K0mostron by Hervé Leclerc, ALTER WAY

🏷️ Versioning

Versioning adheres to the Semantic Versioning principles. A full list of the available releases is available in the GitHub repository's Release section.

📄 Documentation

Further documentation can be found on the official Kamaji documentation website.

🤝 Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please, check if the issue has been already opened by checking the GitHub Issues section. In case it isn't, you can open a new one: a detailed report will help us to replicate it, assess it, and work on a fix.

You can express your intention in working on the fix on your own. The commit messages are checked according to the described semantics. Commits are used to generate the changelog, and their author will be referenced in it.

In case of ✨ Feature Requests please use the Discussion's Feature Request section.

📝 License

The Kamaji Cluster API Control Plane provider is licensed under Apache 2.0. The code is provided as-is with no warranties.

🛟 Commercial Support

CLASTIX is the commercial company behind Kamaji and the Cluster API Control Plane provider.

If you're looking to run Kamaji in production and would like to learn more, CLASTIX can help by offering Open Source support plans, as well as providing a comprehensive Enterprise Platform named CLASTIX Enterprise Platform, built on top of the Kamaji and Capsule project (now donated to CNCF as a Sandbox project).

Feel free to get in touch with the provided Contact form.