Open tylerfontaine opened 4 years ago
In addition to increasing a volume size, adding new volumes to either existing mount points, similar to LVM extensions, and new volumes to new mount points, would be very useful.
When you talk about a workaround that DOESN'T involve recreating the statefulset, what do you mean?
I think the only real option for this is to manually edit the existing PVCs to resize them, run kubectl delete sts --cascade=false <timescale-sts-name>
to delete your statefulset, update the PVC sizes in your helm values, and then trigger a helm upgrade
with those new values.
We run Timescale on EKS. Here are the steps that we follow.
Is this a supported way to grow the volume?
We run Timescale on EKS. Here are the steps that we follow.
- Modify the EBS volumes.
- Update the PVC
- Delete the statefulset
- Update the volume definitions in the values.yaml and deploy helm chart to re-create the sts using the existing PVC
Is this a supported way to grow the volume?
Steps 1 and 2 can be collapsed into a single step if you configure the storageclass with allowVolumeExpansion: true
. See this blog: https://kubernetes.io/blog/2018/07/12/resizing-persistent-volumes-using-kubernetes/
Any update here or a way to resize without downtime ?
@Nainterceptor since k8s 1.15 (technically before, but that's when it was enabled by default) you can resize an in-use pvc, provided your underlying driver supports it: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim
As far as I am aware, you still need to step through recreating the statefulset object though, since the volumeClaimTemplates
section does not support resizing. If you delete the statefulset with --cascade=false
you won't have downtime from this.
If your volume driver DOESN'T support resizing, you still won't have downtime if you are running an HA postgresql cluster. You can remove the pods one by one to avoid that.
Hello, FYI, we've documented pvc expansion process, and --cascade=ophan
is not working in our workflow. Helm instance is created via terraform, and if we delete statefulset without pods, the subsequent terraform apply
will fail with message Error: "timescaledb" has no deployed releases
I don't know why, but no issue without this cascade option.
Due to upstream constraints, pvcs can't be resized in stateful sets, as outlined here: https://github.com/timescale/timescaledb-kubernetes/issues/120
But if there is a workaround to resize the underlying volumes without resizing the claims OR to recreate the statefulset or similar, this would be a great addition to the Admin Guide, allowing for more flexibile deployments via the helm chart.