search

josenk / terraform-provider-esxi

Terraform-provider-esxi plugin
GNU General Public License v3.0
540 stars 154 forks source link

Import configuration and thin provision disks #121

Closed diego-maravankin closed 4 years ago

diego-maravankin commented 4 years ago

Hi all! First of all, I'd like to thank all the devs for this provider, it is really useful

I am opening this issue as a question, since I am having an issue with the esxi provider.

I imported my current configuration using terraform import command to start managing my existing ESXi hosts with terraform.

The problem is that after importing and saving the configuration to my terraform file, when I execute terraform plan the plan is to destroy all vms and create them again, because of the boot_disk_type parameter.

Since my virutal disk was not defined, I created the esxi_virtual_disk resource, but the issue persists.

I would appreciate you pointed me in the right direction here. You can find my tf file attached and the output of terraform show and terraform plan.

Thanks in advance, Diego

virtualmachines.txt terraform_plan.txt terraform_show.txt

josenk commented 4 years ago

Thanks for the feedback. I'm the dev. Well, at least 99.9% of it... :-)

Please verify that you are trying to import when the source system is powered off.. Please refer the Known issues in the README.

diego-maravankin commented 4 years ago

Hi @josenk! Thanks for the super-fast response. I totally missed that in the readme file.

I'll close this issue and open a new if I need further help after testing.

diego-maravankin commented 4 years ago

@josenk one question regarding this, can you point me to the proper section of code to check this? I am working on a production evironment and cannot shutdown the hosts.

I am not familiar with go, but maybe I can help in finding a solution.

josenk commented 4 years ago

You don't really need to know "go" to help with this... If you can run an esxi command to get the disk type while it's powered on, just let me know the commands and the expected output.

However, if you're interested in the commands I run to get the disk type, it's here.

https://github.com/josenk/terraform-provider-esxi/blob/master/esxi/virtual-disk_functions.go#L177-L195

As you can see, I use vmkfstools and grep some information from the output. I found this information using google. Maybe there's a better way now?

To update the plugin, my only requirement is that it needs to be a built-in command in all currently supported esxi versions. (no 3rd party utilities)

diego-maravankin commented 4 years ago

I did some googling, and found that the vmdk file is actually a text file that contains the disks metadata.

Here you can see a cat of a thin provisioned and a thick provisioned disk: Thick provisioned

$ cat monitor02_2.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=fffffffe
parentCID=ffffffff
createType="vmfs"

# Extent description
RW 2097152 VMFS "monitor02_2-flat.vmdk"

# The Disk Data Base 
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "512"
ddb.geometry.heads = "128"
ddb.geometry.sectors = "32"
ddb.longContentID = "47c1e5b4bd882e3eab380114fffffffe"
ddb.uuid = "60 00 C2 9e 8e 8e 09 a1-63 d7 47 fd a2 df cf 38"
ddb.virtualHWVersion = "14"

Thin Provisioned

$ cat monitor02.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=b7a3f684
parentCID=ffffffff
createType="vmfs"

# Extent description
RW 83886080 VMFS "monitor02-flat.vmdk"

# The Disk Data Base 
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "5221"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "3f5eb57be425e98ce35e70b5b7a3f684"
ddb.thinProvisioned = "1"
ddb.toolsInstallType = "4"
ddb.toolsVersion = "10346"
ddb.uuid = "60 00 C2 95 d8 5d 79 8f-02 53 47 10 58 82 5c 55"
ddb.virtualHWVersion = "14"

As you can see, the thin provisioned disk has the ddb.thinProvisioned option set to 1.

so to determine is a disk is thin or thick provisioned you could use the command: grep -q thinProvisioned file.vmdk && echo true

Running it with the previos files:

$ grep -q thinProvisioned monitor02_2.vmdk && echo true
$ grep -q thinProvisioned monitor02.vmdk && echo true
true

The only problem I see with this approach is that when disk is thick provisioned, you cannot determine if it is eager zeroed or lazy zeroed. I'll keep looking into this.

josenk commented 4 years ago

OK, it seems that will allow us to determine if it's thin. Have you found a way to determine if it's Thick or Eager Zero Thick?

My google search also shows there have been inconsistencies with this variable... Some admins have claimed that it's not accurate, or has changed after backups/vmotion. Also NFS datastores have different thin/thick rules. I don't have NFS datastores, so I cannot really test any of those combinations.

http://www.cosonok.com/2011/09/vmware-nfs-and-thin-provisioning.html

One other thing that makes me hesitant to change the code is that I have no access to how vmware really determines the disk type in the UI, command or API... My original search to determine this information has led me to using vmkfstools and so far is has worked as expected. If a changing in the code to determine the disk type produced a different result on someone's system, then they could face the situation of destroying the virtual disks.

BTW: As a work-around to the "powered off" requirement, you can simply update the terraform.tfstate file to specify the boot_disk_type, then terraform show should now show you the infra does not require changes.

diego-maravankin commented 4 years ago

I had the same thought, that I might be inconsistent because it is "just a text file" vs using vmdkfstools that actually reads the virtual disk file. I will look deeper into esxcli and vim-svc commands, so far I haven't found the right command to get the disk type.

diego-maravankin commented 4 years ago

I was playing with vim-svc command, and I found a way of getting the disk configuration for the VM: vim-cmd vmsvc/device.getdevices <vm id>

for each configured disk you'll get a (vim.vm.device.VirtualDisk) section with a backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) subsection, where you can find these two parameters: