Reaserch on VolumeGroup snapshotting and taking advantages of it

[humblec]

Describe the feature you'd like to have

Kubernetes KEP and implementation of Volume Group snapshot is in progress and we have to do some research on how this can be made use of in Ceph CSI project for both RBD and CephFS volumes we provision and control.

Ceph implementation is in progress and we have to list down the consumption of this feature by correlating between upstream API and core Ceph side.

This issue tracks the research and discussion points on the same

[1] https://github.com/ceph/ceph/pull/41572 [2] https://github.com/kubernetes/enhancements/blob/9ab31c34cded378a8ad722c481162ed8999d0cbb/keps/sig-storage/20200212-volume-group.md

[humblec]

Here is the summary:

• [ ] CephFS volumegroup snapshot works in such a way that, The volumeGroup snapshot list is not generated for underlying subvolume groups. That said, while a client request volumegroupsnapshot on CephFS, we get a snapname back but it does not carry list of individual snapshot of PVCs inside it. As per the KEP The volumeGroupSnapshotStatus get updated with the list of snapshots which we are lacking/atm.

Type VolumeGroupSnapshotStatus struct {
...
        // List of volume snapshots
    // +optional
        SnapshotList []VolumeSnapshot
}

• [ ] CephFS subvolumegroup snapshot KEP expects that the PVCs can be brought back or move to the group. CephFS does not provide this capability for now. I see this as a "optional" thing in the KEP though

• [ ] The expectation of volumegroup snapshot is that, other operations on the underlying PVCs are blocked while volumegroupsnapshot is in progress. For example, a clone operation has to be blocked..etc

• [ ] Clone operation of subvolumegroup is not part of the design right now, but it can come in later.

• [ ] Life cycle of volumegroup snapshot is yet to be clarified. Highly likely that VolumeGroup and GroupSnapshot should have independent lifecycles too. But no concrete thoughts about how this is going to be handled.

cc @kotreshhr

Additional thoughts ( mainly for ceph-csi team):

Ceph CSI also has to take some decisions on how can we support volumegroup and subvolume layout to support this feature. against various models like [1] considering below

-- Maintance overhead of many groups ( good/bad) -- Limits of subvolumegroups we can have -- Limit of subvolumes in a group -- snap creation and restore operation time in different models...etc

[1]

• One subvolumeGroup per subvolume layout
• One subvolumegroup for a limited subvolume count ( say 10 )..
• When to take the decision of new volumegroup creation...etc

[humblec]

[update] more discussions are happening on the KEP about individual snapshot list ..etc Ref:

https://github.com/kubernetes/enhancements/pull/1551#discussion_r687683605

https://github.com/kubernetes/enhancements/pull/1551#issuecomment-878805651

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[Madhu-1]

Closing this due to inactivity