karmada-io / karmada

Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
https://karmada.io
Apache License 2.0
4.45k stars 883 forks source link

statefulset副本调度不行 #4546

Open 13567436138 opened 9 months ago

13567436138 commented 9 months ago

What would you like to be added:

Why is this needed:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: ceph-block
spec:
  resourceSelectors:
    - apiVersion: v1
      kind: Service
      name: nginx
  placement:
    clusterAffinity:
      clusterNames:
        - cluster1
        - cluster2 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 5
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "ceph-block"
      resources:
        requests:
          storage: 10Mi
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: ceph-block
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: StatefulSet
      name: web
  placement:
    clusterAffinity:
      clusterNames:
        - cluster1
        - cluster2
    replicaScheduling:
      replicaSchedulingType: Divided
      replicaDivisionPreference: Weighted
      weightPreference:
        staticWeightList:
        - targetCluster:
            clusterNames:
            - cluster1
          weight: 2
        - targetCluster:
            clusterNames:
            - cluster2
          weight: 3

root@karmada-02-a:~# kubectl get pod NAME READY STATUS RESTARTS AGE objectstorage-controller-7dff56f8fc-jvp4w 1/1 Running 1 (24h ago) 46h test 1/1 Running 11 (24h ago) 14d web-0 1/1 Running 0 7m31s web-1 1/1 Running 0 4m14s web-2 1/1 Running 0 3m38s web-3 1/1 Running 0 3m22s web-4 1/1 Running 0 2m34s

root@karmada-03-a:~# kubectl get pod NAME READY STATUS RESTARTS AGE test 1/1 Running 5 (24h ago) 9d web-0 1/1 Running 0 2m56s web-1 1/1 Running 0 2m27s web-2 1/1 Running 0 103s web-3 1/1 Running 0 66s web-4 1/1 Running 0 28s

两个集群调度了同样的副本,而不是weighted

13567436138 commented 9 months ago

Error from server (BadRequest): error when creating "sts-nginx.yaml": StatefulSet in version "v1" cannot be handled as a StatefulSet: strict decoding error: unknown field "spec.ordinals" kube-apiserver版本不够高,手动改image可以么,不会冲突把

liangyuanpeng commented 9 months ago

Ordinals feature of Statefulset is Beta at kubernetes v1.27, so if you want to use this feature by default which requires k8s v1.27, I think you have found the solution. @13567436138

liangyuanpeng commented 9 months ago

/remove-kind feature /kind question

RainbowMango commented 9 months ago

Seems that the issue is that the replicas (5) didn't split as expected. Am I right? What @liangyuanpeng is talking about is the start ordinal feature.