Destroy AWS cluster: Add option to preserve certain resources

[cwilkers]

Version

$ openshift-install version
openshift-install-linux-amd64 v0.16.1

Platform (aws|libvirt|openstack):

aws

What happened?

I employed Velero to make backups to S3 of a cluster, then destroyed and re-installed it. PV snapshots were made by Velero in the form of AWS EC2 ELB volume snapshots, but on restoring to the new cluster, it became apparent they were destroyed by the openshift-install destroy cluster command.

What you expected to happen?

I would like an option to preserve snapshots for use in another cluster.

How to reproduce it (as minimally and precisely as possible)?

Minimal steps:

• Create pvc on AWS cluster
• Use a simple pod, e.g. busybox to write a file to the volume
• Snapshot the ebs volume
• Destroy the cluster

Anything else we need to know?

Simply a feature request

References

None

[wking]

The AWS destroyer looks up resources to reap by tag, so if you want to preserve a copy beyond destroy cluster, remove the kubernetes.io/cluster/...: owned tag from your copy.

[staebler]

Why is the snapshot getting tagged? I would be surprised if the tags are inherited from the ebs volume.

[wking]

Can you attach your .openshift_install.log after a destroy cluster? That will show us how the installer found the assets it's removing. Maybe were walking to the snapshot from a tagged resource, if the snapshot itself is untagged?

[cwilkers]

@wking Unfortunately, the logs for the cluster were not available. I would hazard a guess that the snapshots are being tagged because they are created from within the cluster by Velero creating a VolumeSnapshot.

[staebler]

If there are snapshots that are managed by OpenShift that the end user wants to preserve, the end user should also be able to copy the snapshot. That may be a better story than explaining to the end user about tags and which ones to remove. It keeps with the idea that everything managed by OpenShift is cleaned up when the OpenShift cluster is deleted.

Either the tag approach or the copy approach both seem better to me than adding options to the destroyer to preserve certain things.

[cwilkers]

I agree with that assessment, @staebler.

For what it's worth, it looks like Velero is calling the AWS API directly to create the snapshots, and assigning the cluster tags explicitly according to the volumes' tags. Perhaps Velero would be the right place to request this kind of feature.

Thanks, I'll close the issue.