Open vistasunil opened 1 year ago
Hi @vistasunil
In theory it should not make a difference for K8up what provider is behind the RWO PVCs. K8up tries to place the backup pods in such a way that, if they are mounted, it will run on the same node, thus allowing access to the data.
Openebs also seems to be using ISCSI under the hood, so it should behave the same as your other RWO PVCs. However we have not explicitly tested it with openebs. So there might be a chance of some implementation specific issue at hand.
One thing I could think of that lead to this issue: Is it possible that the pod mounting the openebs PVC has moved to another k8s node while the backup was running? If that happens, then the scheduled backup pod can't access the PVC anymore.
@Kidswiss I was also expecting same but backup did not work for openebs PVCs. Also, there was no pod movement during the backup run. So last statement do not stands valid case here. Can you please test the same at your end and let us know what is missing here?
@Kidswiss Any update on this?
if you configure your openebs-zfs-storageclass with parameter: shared: "yes" it works fine.
for example:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: openebs-zfs
parameters:
recordsize: "128k"
compression: "on"
dedup: "off"
fstype: "zfs"
atime: "off"
poolname: "zpool/pvcs"
shared: "yes"
provisioner: zfs.csi.openebs.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
see description here: https://github.com/openebs/zfs-localpv/blob/develop/docs/storageclasses.md#storageclass-for-sharing-the-persistence-volumes
Description
Hello Team,
I was trying to use k8up to backup RWO PVs for openebs, iscsi and nfs storage classes each. It worked well for iscsi and nfs backed PVs but the backup pod failed with below error while mounting openebs backed PV for backup:
MountVolume.SetUp failed for volume "pvc-e67547c6-fd91-445b-8223-98694b15655e" : rpc error: code = Internal desc = verifyMount: device already mounted at [/var/lib/kubelet/pods/d0e858c5-8bce-4d1d-8d8e-07cf4abf8334/volumes/kubernetes.io~csi/pvc-e67547c6-fd91-445b-8223-98694b15655e/mount /var/lib/kubelet/pods/d0e858c5-8bce-4d1d-8d8e-07cf4abf8334/volume-subpaths/pvc-e67547c6-fd91-445b-8223-98694b15655e/yelb-db/0]
Kindly let us know, if k8up supports backing up openebs backed PVs. If yes, then please share the settings.
Thanks & Regards, Sunil Kumar
Additional Context
No response
Logs
No response
Expected Behavior
k8up should support backing up openebs backed PVs.
Steps To Reproduce
No response
Version of K8up
2.7.1
Version of Kubernetes
v1.24
Distribution of Kubernetes
Rancher