node grouping and grading based on node label selector

[snimje]

What would you like to be added:

1. Node grouping based on node labels
2. Add capability to rank node groups across clusters depending on free resources on the nodes in the group
3. Schedule workload on the cluster which has the node group with minimum used resources. (better placement)

Considering two clusters each with 20 BareMetal nodes. Capacity wise these nodes are sized equally. Two nodes from each clusters have SSD disks. These two nodes are labelled as "disktype=ssd".

Following is the pod's manifest :

apiVersion: v1 kind: Pod metadata: name: nginx-pod namespace: default spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms:

• matchExpressions:
  • key: disktype operator: In values:
    • ssd containers:
      • name: nginx image: nginx imagePullPolicy: IfNotPresent

As you can see this pod will land on specific node labelled as "disktype=ssd". I would like schedule this pod through deployment (replica=1).The workload should run on only one cluster. Many such pod deployments will be scheduled through karmada.

I would like to do this - create a node group based on node labels. determine the node group's free resources ( take into account the availability of resource on the nodes labelled as "disktype=ssd" on every cluster)and schedule the pod on the cluster where the node group has more free resources.

Environment: Karmada version: 1.9.0 Kubernetes version: 1.26.10 Others:

Why is this needed: multiple clusters have node capable to carry out certain tasks only on specific nodes. The nodes can be short listed via node label selector and node affinity. Karmada right now unable to identify the cluster with least used nodes based on node label and selector. This means the workload might land on a stressed cluster node ignoring a better cluster.

[XiShanYongYe-Chang]

Hi @snimje, thanks for your feedback!

How should we represent Node groups in a cluster in Karmada? I'd like to hear your thoughts.