search

kube-hetzner / terraform-hcloud-kube-hetzner

Optimized and Maintenance-free Kubernetes on Hetzner Cloud in one command!
MIT License
2.31k stars 352 forks source link

Issue resizing volume #439

Closed yar1k0v closed 1 year ago

yar1k0v commented 1 year ago

I got an error after resizing PVC for one of the statefulsets in the cluster. Resizing process was:

That's the pretty common way, I suppose. And previously I didn't have any issues on the same cluster. I've also tried to resize it manually on node but it still says same - resize2fs: Permission denied to resize filesystem.

  Warning  FailedMount             46s (x8 over 114s)  kubelet                  MountVolume.Setup failed while expanding volume for volume "pvc-3af31c91-7aa6-4433-b5c5-4a1d720921d8" : Expander.NodeExpand failed to expand the volume : rpc error: code = Internal desc = failed to resize volume: resize of device /dev/disk/by-id/scsi-0HC_Volume_24361779 failed: exit status 1. resize2fs output: resize2fs 1.45.7 (28-Jan-2021)
Filesystem at /dev/disk/by-id/scsi-0HC_Volume_24361779 is mounted on /var/lib/kubelet/plugins/kubernetes.io/csi/csi.hetzner.cloud/1749c6186873d393ca953daf594ff37673e1ecb5bfe6179541729e60e432a445/globalmount; on-line resizing required
resize2fs: Permission denied to resize filesystem
old_desc_blocks = 75, new_desc_blocks = 88

Could anyone please give an advice, thanks a lot.

mysticaltech commented 1 year ago

@yar1k0v That volume is a Hetzner Volume, basically, it's automatically requested from them via the Hetzner CSI and mounted on the node. So IMHO you cannot resize it. However, what you could do is create a new PVC requesting the size you want, mount it to the same node, and copy the data from the old to the new (with rsync for instance).

Also see https://github.com/hetznercloud/csi-driver/issues/8

yar1k0v commented 1 year ago

Unmounting and resizing disks on nodes helped. @mysticaltech thanks for the response