Ability to restore VSI from snapshot consistency group

[karthik0918]

Description

A VSI created using terraform, we would like to restore a VSI retaining the primary IP address, VSI tags, VSI name, VSI Hostname. Restore from a snapshot consistency group snapshot (includes boot and data) Get the VSI to a previous state by retaining IP address / Network interface, VSI tags, VSI name, VSI Hostname Needs to be available for customers to consume via a deployable architecture and/or IBM Cloud Schematics

New or affected modules

https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone-vsi/blob/ca728d0bbcb2fae1218ddcac7af1bd3e1ad25ba6/main.tf#L92

By submitting this issue, you agree to follow our Code of Conduct

[in-1911]

We had a similar scenario where a number of VSIs have been deployed via landing zone and we needed to be able to restore a VSI's storage volumes from snapsnots on demand. After some discussions on the topic, I believe this operation should be done outside of the context of the terraform that provisions the VSIs to begin with.

• Landing zone usually provisions multiple VSIs using the same configuration element - e.g. across several zones or multiple instances into a zone. When one of them needs to get restored, it's not feasible to "carve" it out of the configuration and specify the source snapshots for its volumes in the original configuration that used to deploy it.
• Even if there is only one instance in the landing zone configuration, changing the original configuration now ties the environment deployment to a snapshot which will be used from now on for that instance, so the snapshot has to be retained forever.
• If the restored volumes and the new VSI (you cannot attach a new boot volume to an existing VSI) are created in a different terraform state, now you have the original state that is no longer in sync and a new "parallel" one for the restored instance.

The only option to keep the original terraform configuration and state consistent and not tied to the specific snapshot is to import the new resoruces (VSI, volumes, etc) into the original terraform state after the restore has been done with creating a new VSI. But I think that may be too risky.

[toddgiguere]

@in-1911 can you reach out to me through internal slack? I might have some questions for you as we dive deeper into this, based on your experiences?

[toddgiguere]

We have added this feature to this module in order to facilitate the following use-cases asked for by multiple consumers:

1. Deploy and manage* new VSIs based on previously created snapshots
2. Recreate/rollback (destroy/create) previously deployed and managed* VSI instances
3. Specify a "Snapshot Consistency Group Id" to determine snapshot images for boot and attached volumes
4. When VSIs are recreated for any reason (including snapshot rollback) all attributes are retained, including hostnames, IPs, floating IPs, network interfaces, security groups, attached volumes, etc.

* - managed by the Terraform configuration and state using this module

This is being accomplished with the following changes made to both IBM Cloud Platform and the terraform-ibm-landing-zone-vsi module.

IBM Cloud Platform Changes have been made to creation of Snapshot Consistency Groups to add service tags to snapshot group members in order to correctly identify the boot and attached volumes of the VSI instance the group was created from. This change has been successfully deployed to all IBM Cloud Gen2 regions.

terraform-ibm-landing-zone-vsi module The following changes were made to the existing terraform-ibm-landing-zone-vsi module:

• new input variable manage_reserved_ips, if true then reserved IPs are managed by terraform and mapped by hostname to ensure they are retained on VSI recreation
• new input variable use_static_boot_volume_name, if true will override the default behavior of a random generated VSI boot volume name in favor of a name that will persist between VSI recreation
• new input variables boot_volume_snapshot_id and block_storage_volumes.snapshot_id allowing for specific snapshot association with volumes
• new input variable snapshot_consistency_group_id along with new logic to assign snapshot IDs based on service tags provided from consistency group

[in-1911]

@toddgiguere - for the managed reserved IPs - is there a way to specify the desired reserved IPs per subnet? Also - if there are multiple VSIs per subnet - is it just going to assign some arbitrary IPs initially and just keep them the same? It would be nice to be able to give either a specific range or specific IPs.

[toddgiguere]

@in-1911 I am using the Terraform provider https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/is_subnet_reserved_ip to reserve IPs, but I am not including the optional address field to assign a specific IP from that subnet, so it will automatically pick the next available IP based on the subnet CIDR block. I have set auto_delete to false so that the reserved IP is kept even when its assigned resource (in this case a VSI) is destroyed.

The terraform provider only takes a specific IP, not a range, and I didn't have a plan for how the consumer would supply specific IP address to the many VSIs created in the module, which are dynamically created and not known when supplying input variables.

[toddgiguere]

Pull request has been opened: https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone-vsi/pull/685

[in-1911]

@toddgiguere - the problem with relying on IP allocation is that it only works when only the VSI resource is recreated. And you will not know the IPs until you deploy - so anything else linked to those IPs will have to be updated (e.g. scripts configuring the applications). If you undeploy the whole module (or if somehow a set of resources is recreated, e.g. if a subnet or VPC or a resource group forces recreate), it will destroy the reserved IP resources and then pick the new IPs at random.

I had to use my own fork of the VSI module to be able to explicitly assign IP addresses so that they do not change no matter what is being redeployed. I ended up using an array of objects (one per subnet) in each VSI object so that each subnet a VSI is created in got its own IP.

I did not have a case where I would have to specify IPs for multiple VSIs in the same subnet - I think in that case I would go with an auto-increment starting from the first one. But usually when you have multiple instances per subnet it become less critical to always have exactly the same IPs.

[toddgiguere]

Solution released July 30, 2024: https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone-vsi/releases/tag/v4.1.0