Open ChenYi015 opened 1 month ago
@pan3793 @RexXiong PTAL, thanks.
To make it can be reviewed and merged easily, I had created a new PR #2654 to do some values renaming. After it has been merged, I will do a rebase.
For my opinion, volumeClaimTemplates is needed but host path and empty path is more general for most of situations. It's better to add it as an option instead of the only choice.
For my opinion, volumeClaimTemplates is needed but host path and empty path is more general for most of situations. It's better to add it as an option instead of the only choice.
@lianneli Thanks for the advice. Though, I think volumes with type hostPath
and emptyDir
are still supported by [master|worker].volumes
. Before, the configuration is like this:
volumes:
master:
- mountPath: /mnt/celeborn_ratis
hostPath: /mnt/celeborn_ratis
type: hostPath
capacity: 100Gi
worker:
- mountPath: /mnt/disk1
hostPath: /mnt/disk1
type: hostPath
diskType: SSD
capacity: 100Gi
After, it will be like this:
master:
volumes:
- name: celeborn-ratis
hostPath:
path: /mnt/celeborn_ratis
volumeMounts:
- name: celeborn-ratis
mountPath: /mnt/celeborn_ratis
worker:
volumes:
- name: disk1
hostPath:
path: /mnt/disk1
volumeMounts:
- name: disk1
mountPath: /mnt/disk1
celeborn:
celeborn.worker.storage.dirs=/mnt/disk1:disktype=SSD:capacity=100Gi
The difference is that we need to configure celeborn.worker.storage.dirs
manually if we use hostPath
or emptyDir
.
If we want to use volumeClaimTemplates
, suppose ssd-storage-class
is a valid storage class in your Kubernetes cluster that provided by a cloud provider or defined by yourself that can be used to provide SSD cloud disks dynamically, the configuration will be like as follows:
master:
volumeClaimTemplates:
- metadata:
name: celeborn-ratis
annotations:
celeborn.apache.org/disk-type: SSD
celeborn.apache.org/disk-capacity: 100Gi
spec:
accessModes:
- ReadWriteOnce
resources:
request:
storage: 100Gi
limits:
storage: 100Gi
storageClassName: ssd-storage-class
volumeMounts:
- name: celeborn-ratis
mountPath: /mnt/celeborn_ratis
worker:
volumeClaimTemplates:
- metadata:
name: disk1
annotations:
celeborn.apache.org/disk-type: SSD
celeborn.apache.org/disk-capacity: 100Gi
spec:
accessModes:
- ReadWriteOnce
resources:
request:
storage: 100Gi
limits:
storage: 100Gi
storageClassName: ssd-storage-class
volumeMounts:
- name: disk1
mountPath: /mnt/disk1
The annotations celeborn.apache.org/disk-type
and celeborn.apache.org/disk-capacity
are used by Helm to render celeborn.worker.storage.dirs
in configmap.
When we install the Helm chart, SSD cloud disks will be dynamically provisioned by cloud providers.
This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.
What changes were proposed in this pull request?
Make helm chart more customizable:
All master related configurations are prefixed with
master
, e.g.master.replicas
,master.volumeClaimTemplates
,master.volumeMounts
.All worker related configurations are prefixed with
worker
, e.g.worker.replicas
,worker.nodeSelector
,worker.affinity
.Add support for
labels
,env
,envFrom
andvolumeClaimTemplates
.Bump chart
appVersion
to0.5.1
.Why are the changes needed?
By using volumeClaimTemplates in StatefulSet, we can support various types of storage backend.
Does this PR introduce any user-facing change?
Yes.
How was this patch tested?
Test locally.