[BUG] addon pod schedule failed: didn't match pod anti-affinity rules with kubeblocks

Describe the bug

kbcli version
Kubernetes: v1.27.4+k3s1
KubeBlocks: 0.8.3

To Reproduce Steps to reproduce the behavior:

create crd

kubectl create -f https://github.com/apecloud/kubeblocks/releases/download/v0.8.3/kubeblocks_crds.yaml

install kubeblocks distributed on 3 different nodes


helm repo add kubeblocks https://apecloud.github.io/helm-charts

helm upgrade -i kubeblocks kubeblocks/kubeblocks -n kb-system --create-namespace --version 0.8.3 -f -<<EOF replicaCount: 3 image: tag: 0.8.3 extraLabels: app.kubernetes.io/component: "kubeblocks" tolerations:

key: kb-controller operator: Equal value: "true" effect: NoSchedule affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100 preference: matchExpressions:
  - key: kb-controller operator: In values:
    - "true" podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector: matchLabels: app.kubernetes.io/instance: kubeblocks app.kubernetes.io/component: "kubeblocks" topologyKey: kubernetes.io/hostname dataProtection: image: tag: 0.8.3 affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution:
  - weight: 100 preference: matchExpressions:
    - key: kb-controller operator: In values:
    - "true" podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution:
  - labelSelector: matchLabels: app.kubernetes.io/instance: kubeblocks app.kubernetes.io/component: "dataprotection" topologyKey: kubernetes.io/hostname extraLabels: app.kubernetes.io/component: "dataprotection" EOF
```
3. See error
```
    kubectl get pod -n kb-system NAME READY STATUS RESTARTS AGE install-redis-addon-pcmqs 0/1 Pending 0 173m install-kafka-addon-z4dx4 0/1 Pending 0 173m install-postgresql-addon-z9wkc 0/1 Pending 0 173m install-apecloud-mysql-addon-2ssm8 0/1 Pending 0 173m install-clickhouse-addon-8whjm 0/1 Pending 0 173m install-snapshot-controller-addon-fwhg6 0/1 Pending 0 173m install-mongodb-addon-glkfl 0/1 Pending 0 173m install-pulsar-addon-682kd 0/1 Pending 0 173m kubeblocks-75c5847c8d-2m5pk 1/1 Running 0 173m kubeblocks-75c5847c8d-5hd68 1/1 Running 0 173m kubeblocks-75c5847c8d-hvmls 1/1 Running 0 173m kubeblocks-dataprotection-59d9446c4f-wp7lh 0/1 CrashLoopBackOff 25 (4m50s ago) 173m kubeblocks-dataprotection-59d9446c4f-zqpjr 1/1 Running 28 (2m47s ago) 173m kubeblocks-dataprotection-59d9446c4f-7kc9j 0/1 CrashLoopBackOff 28 (43s ago) 173m
```
describe Pending pod
```
    kubectl describe pod -n kb-system install-apecloud-mysql-addon-2ssm8
    Name: install-apecloud-mysql-addon-2ssm8 Namespace: kb-system Priority: 0 Node: Labels: app.kubernetes.io/managed-by=kubeblocks batch.kubernetes.io/controller-uid=351ab7be-4fdb-4d7b-ac37-2f0e4cf7d8cd batch.kubernetes.io/job-name=install-apecloud-mysql-addon controller-uid=351ab7be-4fdb-4d7b-ac37-2f0e4cf7d8cd extensions.kubeblocks.io/addon-name=apecloud-mysql job-name=install-apecloud-mysql-addon Annotations: Status: Pending IP:
    IPs: Controlled By: Job/install-apecloud-mysql-addon Init Containers: copy-charts: Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-charts:0.8.3 Port: Host Port: Command: sh -c cp /charts/* /mnt/charts Limits: cpu: 0 memory: 0 Requests: cpu: 0 memory: 0 Environment: Mounts: /mnt/charts from charts (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-64cks (ro) Containers: helm: Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.3 Port: Host Port: Command: helm Args: upgrade --install $(RELEASE_NAME) /charts/apecloud-mysql-0.8.0-beta.7.tgz --namespace $(RELEASE_NS) --create-namespace --atomic --cleanup-on-fail --wait --insecure-skip-tls-verify Limits: cpu: 0 memory: 0 Requests: cpu: 0 memory: 0 Environment: RELEASE_NAME: kb-addon-apecloud-mysql RELEASE_NS: kb-system CHART: file:///apecloud-mysql-0.8.0-beta.7.tgz Mounts: /charts from charts (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-64cks (ro) Conditions: Type Status PodScheduled False Volumes: charts: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
    SizeLimit: kube-api-access-64cks: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: Tolerations: kb-controller=true:NoSchedule node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message
    
    Warning FailedScheduling 13m default-scheduler 0/3 nodes are available: 3 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod.. Warning FailedScheduling 8m7s default-scheduler 0/3 nodes are available: 3 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod..
```
get kubeblocks pod yaml
```
    kubectl get pod -oyaml kubeblocks-75c5847c8d-2m5pk -n kb-system apiVersion: v1 kind: Pod metadata: creationTimestamp: "2024-05-27T03:57:30Z" generateName: kubeblocks-75c5847c8d- labels: app.kubernetes.io/component: kubeblocks app.kubernetes.io/instance: kubeblocks app.kubernetes.io/name: kubeblocks pod-template-hash: 75c5847c8d name: kubeblocks-75c5847c8d-2m5pk namespace: kb-system ownerReferences:
- apiVersion: apps/v1 blockOwnerDeletion: true controller: true kind: ReplicaSet name: kubeblocks-75c5847c8d uid: 5eebfb4f-ebaf-421f-9387-aca9d2a9152d resourceVersion: "1845" uid: 4602de3c-4b2e-4612-8602-2deed277cc1a spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution:
  - preference: matchExpressions:
    - key: kb-controller operator: In values:
    - "true" weight: 100 podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution:
  - labelSelector: matchLabels: app.kubernetes.io/component: kubeblocks app.kubernetes.io/instance: kubeblocks topologyKey: kubernetes.io/hostname

get addon pod yaml

kubectl get pod -oyaml install-apecloud-mysql-addon-2ssm8 -n kb-system apiVersion: v1 kind: Pod metadata: creationTimestamp: "2024-05-27T03:58:24Z" finalizers:

batch.kubernetes.io/job-tracking generateName: install-apecloud-mysql-addon- labels: app.kubernetes.io/managed-by: kubeblocks batch.kubernetes.io/controller-uid: 351ab7be-4fdb-4d7b-ac37-2f0e4cf7d8cd batch.kubernetes.io/job-name: install-apecloud-mysql-addon controller-uid: 351ab7be-4fdb-4d7b-ac37-2f0e4cf7d8cd extensions.kubeblocks.io/addon-name: apecloud-mysql job-name: install-apecloud-mysql-addon name: install-apecloud-mysql-addon-2ssm8 namespace: kb-system ownerReferences:
apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: install-apecloud-mysql-addon uid: 351ab7be-4fdb-4d7b-ac37-2f0e4cf7d8cd resourceVersion: "1788" uid: 404bbb44-d680-4900-9eec-c2660e7aa2cd spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution:
- preference: matchExpressions:
  - key: kb-controller operator: In values:
    - "true" weight: 100 podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector: matchLabels: app.kubernetes.io/component: kubeblocks app.kubernetes.io/instance: kubeblocks topologyKey: kubernetes.io/hostname



**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: [e.g. iOS]
 - Browser [e.g. chrome, safari]
 - Version [e.g. 22]

**Additional context**
Add any other context about the problem here.

apecloud / kubeblocks

[BUG] addon pod schedule failed: didn't match pod anti-affinity rules with kubeblocks #7426