Restore failes for PVCs that have an ownerReference

[Sediket]

Describe the bug Re-creating: https://github.com/vmware-tanzu/velero/issues/7862

I'm implementing velero as our backup solution and testing backups. I backed up a helm install of grafana's loki (https://github.com/grafana/loki/tree/main/production/helm/loki) and I noticed the PVCs have an ownerReference set to the loki-backend StatefulSet. So on restore the PVC will not be created referencing the ownerReference, same for all the PVCs in the StatefulSet. There are PVCs that don't have an ownerReference and they restore correctly.

As I understand Velero should be removing the metaData to include the ownerReference but it doesn't seem to be happening for some reason.

I also tested this by re-creating my storage class so it doesn't have to re-map the storage class on restore, but I still get the same result. The PVCs are empty and error, referencing the ownerReference is not present in the namespace.

Details in the attached issue.

[swiftlee]

Hello! I am working with @Sediket on this issue. I'm fairly new to Velero and its plugins, so I may be incorrect.

It looks like the the only metadata that is removed from PVCs at restore is health annotations as shown here: https://github.com/vmware-tanzu/velero-plugin-for-vsphere/blob/main/pkg/plugin/restore_pvc_action_plugin.go#L97

It appears that all other metadata removal is omitted for PVCs specifically, and it does seem to override the parent behavior from Velero.

Given this, PVCs are keeping the ownerReference to their corresponding StatefulSet. However, when the PVCs attempt to restore they fail since no such owner exists in the mapped namespace.

[xing-yang]

Can you provide kubectl describe of the StatefulSet and PVCs that failed to be restored and the ones can be restored?

[Sediket]

Yes, here they are successfulPVC.log successfulSTS.log failedPVC.log failedSTS.log

[xing-yang]

Alright. Looks like this is a bug.