search

zilliztech / milvus-helm

Apache License 2.0
58 stars 43 forks source link

Allow setting nodeSelectors and tolerations for pulsar components #124

Open aliabbasjaffri opened 2 months ago

aliabbasjaffri commented 2 months ago

For nodes with custom labels and taints, there is currently no way to natively set nodeSelectors and tolerations on all the pods created by pulsar in a kubernetes cluster

aliabbasjaffri commented 2 months ago

This can be done by bumping up pulsar version. The current set version is 2.7.8 which does not support tolerations in the helm chart, however version 3.5.0 supports tolerations.

aliabbasjaffri commented 2 months ago

However i'm not sure what would be the side-effects of bumping the version.

However, pulsar version 2.7.8 was updated 3 years ago, any reason why it has not been updated?

haorenfsa commented 2 months ago

pulsar v3 chart doesn't support pulsar v2 image. So we're still using v2 for backward compatibility. https://github.com/milvus-io/milvus/issues/30812

As a walk around, You can deploy pulsar with v3 chart and then configure milvus to use that pulsar. Or you can also use milvus-operator.

aliabbasjaffri commented 2 months ago

Is there any timeline to when pulsar v3 would be supported by the helm chart?

haorenfsa commented 2 months ago

@aliabbasjaffri It will probably be in October. We are still designing a solution to introduce it without affecting compatibility.

colinbjohnson commented 2 months ago

@aliabbasjaffri although we would love to utilize pulsar version 3 we are using the existing milvus helm charts with pulsar and nodeSelectors.

proxy:
  nodeSelector:
    kubernetes.io/arch: amd64
  replicaCount: 1
  podMonitor:
    enabled: false
  resources:
    requests:
      memory: 2048Mi
      cpu: 1
  service:
    type: ClusterIP
  ports:
    pulsar: 6650
  configData:
    PULSAR_MEM: >
      -Xms2048m -Xmx2048m
    PULSAR_GC: >
      -XX:MaxDirectMemorySize=2048m
aliabbasjaffri commented 2 months ago

@colinbjohnson that is strange, how do you pass nodeSelector to pulsar_metadata component which then passes those values to bookkeeper-cluster-init and pulsar-cluster-init, given any of those are not supported by pulsar helm chart version 2.7.8.

haorenfsa commented 1 month ago

milvus-operator now supports pulsar-v3 chart, check this sample: https://github.com/zilliztech/milvus-operator/blob/main/config/samples/milvus_pulsar_v3.yaml

milvus-helm chart will support it later.

haorenfsa commented 1 month ago

related to: https://github.com/milvus-io/milvus/issues/30812