kubewharf / kubeadmiral

Multi-Cluster Kubernetes Orchestration
Apache License 2.0
658 stars 89 forks source link

feat: add new hpa aggregator for centralized-hpa mode #293

Closed JackZxj closed 6 months ago

JackZxj commented 7 months ago

A new HPA aggregator component is introduced to implement the capabilities of centralized HPA. It is essentially a lightweight agent and an aggregator that provides aggregated information for HPA controllers.

Only when an HPA resource has the label kubeadmiral.io/centralized-hpa-enabled: true, the HPA controller can read the HPA resource through the HPA aggregator. In this way, any kind of HPAs can be supported non-intrusively for our centralized HPA mode, and any HPA controller the user has been used can be easily reuse in federation without multi-cluster adaptation.

The HPA aggregator will aggregate the pod information of all member clusters (in order to prevent duplicate names, a name will be regenerated with the cluster name). It also implements metrics.k8s.io and custom.metrics.k8s.io, from which the HPA controller can obtain aggregation pod metric information or custom metric information to provide data support for auto scaling workloads.

The HPA controller's scale up/scale down operations for the workload will also be forwarded to the KubeAdmiral apiserver. How these replicas are ultimately scheduled still depends on the kubeadmiral controller.

You can use make local-up to try it.

Note: HPA aggregator will use the permissions carried in the request to forward resource requests, so authorization is still required. However, the only exception is that aggregated pod information currently cannot be effectively authorized, so requests for pods only support read-only operations and do not support subresources of pod.