"cannot find disk device" when orphaned disks are destroyed before the virtual machine in the same operation

Community Guidelines

[X] I have read and agree to the HashiCorp Community Guidelines .
[X] Vote on this issue by adding a 👍 reaction to the original issue initial description to help the maintainers prioritize.
[X] Do not leave "+1" or other comments that do not add relevant information or questions.
[X] If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform

v1.3.6

Terraform Provider

v2.2.0

VMware vSphere

v7.0.3

Description

When a single Terraform module creates a vsphere_virtual_machine and add an orphaned_disk (through the vSphere Container Storage Plug-in in my case), then, when the module is destroyed, it destroys the orphaned_disk and the vsphere_virtual_machine resource in the same operation. Since the orphaned disk is destroyed before the virtual machine, the Delete operation does not find the disk and return a cannot find disk device error and does not destroy the vsphere_virtual_machine resource.

The error should be silently ignored if the orphaned disk is already destroyed or detached since the infrastructure is already at the desired state.

The relevant code seems to be here:

func (r *NetworkInterfaceSubresource) Delete(l object.VirtualDeviceList) ([]types.BaseVirtualDeviceConfigSpec, error) {
    log.Printf("[DEBUG] %s: Beginning delete", r)
    vd, err := r.FindVirtualDevice(l)
    if err != nil {
        return nil, fmt.Errorf("cannot find network device: %s", err)
    }

Affected Resources or Data Sources

resource/vsphere_virtual_machine

Terraform Configuration

resource "vsphere_virtual_machine" "main" {
  name             = local.name
  datastore_id     = local.datastore_id
  memory           = 1024
  num_cpus         = 1
  resource_pool_id = local.resource_pool_id

  cdrom {
    client_device = true
  }

  clone {
    template_uuid = local.template_uuid
  }

  disk {
    controller_type = "scsi"
    datastore_id    = local.datastore_id
    label           = "disk0"
    size            = 10
  }

  network_interface {
    network_id = local.network_id
  }

  lifecycle {
    ignore_changes = [disk]
  }
}

Debug Output

vsphere_virtual_machine.main: Destroying... [id=4226126e-04dc-0441-2665-f6d8e8e92ca5]
2022-12-27T09:41:36.782-0500 [INFO]  Starting apply for vsphere_virtual_machine.main
2022-12-27T09:41:36.782-0500 [DEBUG] vsphere_virtual_machine.main: applying the planned Delete change
2022-12-27T09:41:36.794-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] vsphere_virtual_machine (ID = 4226126e-04dc-0441-2665-f6d8e8e92ca5): Performing delete: timestamp=2022-12-27T09:41:36.794-0500
2022-12-27T09:41:36.794-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] Locating virtual machine with UUID "4226126e-04dc-0441-2665-f6d8e8e92ca5": timestamp=2022-12-27T09:41:36.794-0500
2022-12-27T09:41:36.794-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] Using SearchIndex to look up UUID "4226126e-04dc-0441-2665-f6d8e8e92ca5": timestamp=2022-12-27T09:41:36.794-0500
2022-12-27T09:41:36.811-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] VM "/DC/vm/test-orphaned" found for UUID "4226126e-04dc-0441-2665-f6d8e8e92ca5": timestamp=2022-12-27T09:41:36.811-0500
2022-12-27T09:41:36.811-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] Fetching properties for VM "/DC/vm/test-orphaned": timestamp=2022-12-27T09:41:36.811-0500
2022-12-27T09:41:36.841-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] Fetching properties for VM "/DC/vm/test-orphaned": timestamp=2022-12-27T09:41:36.841-0500
2022-12-27T09:41:36.863-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:36 [DEBUG] Attempting guest shutdown of virtual machine "/DC/vm/test-orphaned": timestamp=2022-12-27T09:41:36.863-0500
2022-12-27T09:41:46.546-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:46 [DEBUG] DiskDestroyOperation: Beginning destroy: timestamp=2022-12-27T09:41:46.546-0500
2022-12-27T09:41:46.546-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:46 [DEBUG] DiskDestroyOperation: Detaching devices with keep_on_remove enabled: timestamp=2022-12-27T09:41:46.546-0500
2022-12-27T09:41:46.546-0500 [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2022/12/27 09:41:46 [DEBUG] disk.1 (key 2001 at scsi:0:1) (test-orphaned/disk1.vmdk): Beginning delete: timestamp=2022-12-27T09:41:46.546-0500
2022-12-27T09:41:46.565-0500 [ERROR] vertex "vsphere_virtual_machine.main (destroy)" error: disk.1: cannot find disk device: virtual disk with UUID 6000C292-4082-6b5d-fdcc-8a4649e160e5 not found
╷
│ Error: disk.1: cannot find disk device: virtual disk with UUID 6000C292-4082-6b5d-fdcc-8a4649e160e5 not found

Panic Output

No response

Expected Behavior

The Delete function should silently ignore a disk sub-resource that is not found (aka already deleted), and continue with the deletion of the vsphere_virtual_machine resource.

Actual Behavior

The Delete function returns a cannot find disk device error and the vsphere_virtual_machine resource is not destroyed.

Steps to Reproduce

This reproduces the same behavior as if a module were creating a virtual machine and attach a new disk using the vSphere Container Storage Plug-in, and then destroy them:

Create a virtual machine: terraform apply
Add a second disk: govc vm.disk.create -vm "${name}" "${name}/disk1" -size 1G
Save destroy plan: terraform plan -destroy -out destroy.plan
Delete the second disk: govc device.remove -vm "${name}" disk-1000-1
Apply the destroy plan: terraform apply destroy.plan

Environment Details

No response

Screenshots

No response

References

No response

hashicorp / terraform-provider-vsphere