virtual_disk_id doesn't get recognized properly

mj84 commented 4 years ago

Describe the bug The provider assumes that a change is necessary due to a UUID in the virtual_disk_id when using additional VMDKs.

E.g.:

# esxi_guest.myvm will be updated in-place
~ resource "esxi_guest" "myvm" {
      boot_disk_size         = "8"
      boot_disk_type         = "zeroedthick"
      disk_store             = "datastore1"
      guest_name             = "MyVM"
      guest_shutdown_timeout = 20
      guest_startup_timeout  = 60
      guestinfo              = {}
      guestos                = "debian10-64"
      id                     = "39"
      ip_address             = "1.2.3.4"
      memsize                = "1024"
      numvcpus               = "4"
      power                  = "on"
      resource_pool_name     = "/"
      virthwver              = "14"

      network_interfaces {
          nic_type        = "vmxnet3"
          virtual_network = "VM Network"
      }

      ~ virtual_disks {
          slot            = "0:1"
        ~ virtual_disk_id = "/vmfs/volumes/5cd04d8f-2d0e8f21-c545-bc305bf1a350/Terraform/myvm_swap.vmdk" -> "/vmfs/volumes/datastore1/Terraform/myvm_swap.vmdk"
      }
  }

To Reproduce Only VMs with additional VMDKs are affected.

define a esxi_virtual_disk resource
add that esxi_virtual_disk to a VM via virtual_disks
run plan, apply, plan again
behaviour should become visible

Expected behavior The provider should correctly resolve the datastore to its UUID (or the other way round) and know that no change is necessary.

Desktop:

OS: CentOS 7.7.1908
Terraform Version: v0.12.20
Ovftool Version: 4.3.0 (build-7948156)
This Plugin Version v1.6.2

On a side note: The current behaviour always causes the provider when running terraform apply to shutdown my VMs, performing a dummy change, and then powering on my VMs again, which is obviously not ideal.

josenk commented 4 years ago

Please upload your *.tf files...

mj84 commented 4 years ago

Please see this example:

resource "esxi_virtual_disk" "myvm_swap" {
  virtual_disk_disk_store = "datastore1"
  virtual_disk_dir = "Terraform"
  virtual_disk_name = "myvm_swap.vmdk"
  virtual_disk_size = "1"
  virtual_disk_type = "zeroedthick"
}
resource "esxi_guest" "myvm" {
  guest_name = "MyVM"
  disk_store = "datastore1"
  power = "on"
  guestos = "centos7-64"
  numvcpus = "4"
  memsize = "4096"
  virthwver = "14"
  boot_disk_type = "zeroedthick"
  boot_disk_size = "8"
  network_interfaces {
      virtual_network = "VM Network"
      nic_type = "vmxnet3"
  }
  virtual_disks {
      virtual_disk_id = esxi_virtual_disk.myvm_swap.id
      slot = "0:1"
  }
}

josenk commented 4 years ago

1) What do you mean by "due to a UUID in the virtual_disk_id"??? 2) Please explain your "side note"... What "dummy change are you doing that you don't expect it to reboot? 3) I wasn't able to reproduce any of your problems...

[root@dev 12.example2]# terraform_0.12.16 plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # esxi_guest.myvm will be created
  + resource "esxi_guest" "myvm" {
      + boot_disk_size         = "8"
      + boot_disk_type         = "zeroedthick"
      + disk_store             = "DS_3TB"
      + guest_name             = "MyVM"
      + guest_shutdown_timeout = (known after apply)
      + guest_startup_timeout  = (known after apply)
      + guestos                = "centos7-64"
      + id                     = (known after apply)
      + ip_address             = (known after apply)
      + memsize                = "4096"
      + notes                  = (known after apply)
      + numvcpus               = "4"
      + ovf_properties_timer   = (known after apply)
      + power                  = "on"
      + resource_pool_name     = (known after apply)
      + virthwver              = "14"

      + network_interfaces {
          + mac_address     = (known after apply)
          + nic_type        = "vmxnet3"
          + virtual_network = "192.168.1"
        }

      + virtual_disks {
          + slot            = "0:1"
          + virtual_disk_id = (known after apply)
        }
    }

  # esxi_virtual_disk.myvm_swap will be created
  + resource "esxi_virtual_disk" "myvm_swap" {
      + id                      = (known after apply)
      + virtual_disk_dir        = "Terraform"
      + virtual_disk_disk_store = "DS_3TB"
      + virtual_disk_name       = "myvm_swap.vmdk"
      + virtual_disk_size       = 1
      + virtual_disk_type       = "zeroedthick"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

[root@dev 12.example2]# terraform_0.12.16 apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # esxi_guest.myvm will be created
  + resource "esxi_guest" "myvm" {
      + boot_disk_size         = "8"
      + boot_disk_type         = "zeroedthick"
      + disk_store             = "DS_3TB"
      + guest_name             = "MyVM"
      + guest_shutdown_timeout = (known after apply)
      + guest_startup_timeout  = (known after apply)
      + guestos                = "centos7-64"
      + id                     = (known after apply)
      + ip_address             = (known after apply)
      + memsize                = "4096"
      + notes                  = (known after apply)
      + numvcpus               = "4"
      + ovf_properties_timer   = (known after apply)
      + power                  = "on"
      + resource_pool_name     = (known after apply)
      + virthwver              = "14"

      + network_interfaces {
          + mac_address     = (known after apply)
          + nic_type        = "vmxnet3"
          + virtual_network = "192.168.1"
        }

      + virtual_disks {
          + slot            = "0:1"
          + virtual_disk_id = (known after apply)
        }
    }

  # esxi_virtual_disk.myvm_swap will be created
  + resource "esxi_virtual_disk" "myvm_swap" {
      + id                      = (known after apply)
      + virtual_disk_dir        = "Terraform"
      + virtual_disk_disk_store = "DS_3TB"
      + virtual_disk_name       = "myvm_swap.vmdk"
      + virtual_disk_size       = 1
      + virtual_disk_type       = "zeroedthick"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

esxi_virtual_disk.myvm_swap: Creating...
esxi_virtual_disk.myvm_swap: Creation complete after 2s [id=/vmfs/volumes/DS_3TB/Terraform/myvm_swap.vmdk]
esxi_guest.myvm: Creating...
esxi_guest.myvm: Still creating... [10s elapsed]
esxi_guest.myvm: Still creating... [20s elapsed]
esxi_guest.myvm: Still creating... [30s elapsed]
esxi_guest.myvm: Still creating... [40s elapsed]
esxi_guest.myvm: Still creating... [50s elapsed]
esxi_guest.myvm: Still creating... [1m0s elapsed]
esxi_guest.myvm: Still creating... [1m10s elapsed]
esxi_guest.myvm: Still creating... [1m20s elapsed]
esxi_guest.myvm: Still creating... [1m30s elapsed]
esxi_guest.myvm: Still creating... [1m40s elapsed]
esxi_guest.myvm: Still creating... [1m50s elapsed]
esxi_guest.myvm: Still creating... [2m0s elapsed]
esxi_guest.myvm: Still creating... [2m10s elapsed]
esxi_guest.myvm: Still creating... [2m20s elapsed]
esxi_guest.myvm: Creation complete after 2m27s [id=345]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

[root@dev 12.example2]# terraform_0.12.16 plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

esxi_virtual_disk.myvm_swap: Refreshing state... [id=/vmfs/volumes/DS_3TB/Terraform/myvm_swap.vmdk]
esxi_guest.myvm: Refreshing state... [id=345]

------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

[root@dev 12.example2]# terraform_0.12.16 refresh
esxi_virtual_disk.myvm_swap: Refreshing state... [id=/vmfs/volumes/DS_3TB/Terraform/myvm_swap.vmdk]
esxi_guest.myvm: Refreshing state... [id=345]

[root@dev 12.example2]# terraform_0.12.16 plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

esxi_virtual_disk.myvm_swap: Refreshing state... [id=/vmfs/volumes/DS_3TB/Terraform/myvm_swap.vmdk]
esxi_guest.myvm: Refreshing state... [id=345]

------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

[root@dev 12.example2]# terraform_0.12.16 apply
esxi_virtual_disk.myvm_swap: Refreshing state... [id=/vmfs/volumes/DS_3TB/Terraform/myvm_swap.vmdk]
esxi_guest.myvm: Refreshing state... [id=345]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.   (This did not reboot my VM)

mj84 commented 4 years ago

Regarding 1.) In my VMX files, all additional VMDKs are referenced by the datastore's UUID, not the datastore's name, e.g.:

scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.fileName = "/vmfs/volumes/5cd04d8f-2d0e8f21-c545-bc305bf1a350/Terraform/myvm_swap.vmdk"
scsi0:1.present = "true"

My guess is that this is what your terraform provider wants to correct, so that the datastore's name is put there instead of the UUID. This is also what i meant by "dummy change". Terraform tries to change the UUID to the datastore name, but that change is never really applied. Still my machines have to reboot, because my free license doesn't allow any modification while the VM is running.

Could you please check if your VMX files contain the datastore UUIDs as well?

josenk commented 4 years ago

I requested information from you. PLEASE be honest in your replies. Since you did not actually post YOUR terraform files, both you and I are wasting our time... I do not appreciate that.

vmx files can use the uuid or the datastore name. This plugin uses the datastore name. Since you didn't upload your tf files, I can only guess that you are incorrectly using the virtual_disk_store, virtual_disk_dir and virtual_disk_name variables.
Are you trying to use existing vmdk files??? Any resource you want to manage with terraform (the vm guest, or virtual disk for example) must be CREATED by terraform or IMPORTED. You cannot just use an existing resource with terraform because it's not in it's statefile. This is how Terraform works.

I suggest to read the README, go through the examples and read the wiki page to import resources.

If you have any additional questions, please upload YOUR terraform files and full details of the issue.

mj84 commented 4 years ago

I just discovered the following: 1) create and delete a VM snapshot 2) the vmx file now contains the datastore UUID for the additional disks 3) your provider wants to change the virtual_disk id from the datastore UUID to the datastore name

So using the datastore name does not seem to be the most stable choice here.

I was able to reproduce the described issue using the exact terraform code I supplied, and just didn't assume that the creation and deletion of snapshots was responsible for changing the datastore name to the UUID. Accusing me of not being honest is really un-called for.

mj84 commented 4 years ago

I have found the following workaround for this issue: Replace the datastore name with its UUID in the virtual_disk_id in the .tf files, e.g.

virtual_disk_id = "/vmfs/volumes/5cd04d8f-2d0e8f21-c545-bc305bf1a350/Terraform/myvm_swap.vmdk"

instead of

virtual_disk_id = "/vmfs/volumes/datastore1/Terraform/myvm_swap.vmdk"

josenk commented 4 years ago

Terraform manages the state. If creating snapshots changes the vmx file, that is the reason for it to make the correction. That is the expected behavior of terraform.

It's certainly a valid workaround to specify the uuid instead of the diskstore name in this case.

mj84 commented 4 years ago

Terraform manages the state. If creating snapshots changes the vmx file, that is the reason for it to make the correction. That is the expected behavior of terraform.

One could argue about that, since creating and removing a snapshot does not change the state of the vmx file from a VMware perspective. I definitely would agree that a VM with a snapshot is certainly in a difficult state from the provider's point of view.

I assume that the best solution would be if the provider would not care if UUIDs or datastore names are being used, probably by resolving the datastore name to the UUID and detecting if the name/UUID match the desired state.

josenk / terraform-provider-esxi

virtual_disk_id doesn't get recognized properly #99

I requested information from you. PLEASE be honest in your replies. Since you did not actually post YOUR terraform files, both you and I are wasting our time... I do not appreciate that.