Support for hot add CPU and memory is broken

sgaosdgr commented 6 months ago

Community Guidelines

[X] I have read and agree to the HashiCorp Community Guidelines .
[ ] Vote on this issue by adding a 👍 reaction to the original issue initial description to help the maintainers prioritize.
[ ] Do not leave "+1" or other comments that do not add relevant information or questions.
[ ] If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform

v1.7.5, v1.8.1

Terraform Provider

v2.7.0

VMware vSphere

v8.0 u2b

Description

The support for hot add CPU and memory is currently broken. A change to CPU and memory causes an VM to reboot. This is 100% reproducible.

Affected Resources or Data Sources

VM/

Terraform Configuration

#VMware vSphere Provider
provider "vsphere" {  
  #Set of variables used to connect to the vCenter
    vsphere_server = var.vsphere_server 
    user           = var.vsphere_user
    password       = var.vsphere_password

#If you have a self-signed cert
    allow_unverified_ssl = true
  }

#Name of the Datacenter in the vCenter
  data "vsphere_datacenter" "dc" {
    name = "DC1"
  }
#Name of the Cluster in the vCenter
  data "vsphere_compute_cluster" "cluster" {
    name          = "Cluster"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Datastore in the vCenter, where VM will be deployed
  data "vsphere_datastore" "datastore" {
    name          = "vSAN"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Portgroup in the vCenter, to which VM will be attached
  data "vsphere_network" "external_network" {
    name          = "External"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Portgroup in the vCenter, to which VM will be attached
  data "vsphere_network" "internal_network" {
    name          = "Public"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Templete in the vCenter, which will be used to the deployment
  data "vsphere_virtual_machine" "rocky9-template" {
    name          = "rocky9-temp"
    datacenter_id = data.vsphere_datacenter.dc.id
  }

#Set VM parameteres
  resource "vsphere_virtual_machine" "rocky9-vm" {
    name = "rocky91"
    num_cpus = 2
    memory   = 4096
    firmware = "efi"
    cpu_hot_add_enabled = "true"
    memory_hot_add_enabled = "true"
    guest_id = "rockylinux_64Guest"
    resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
    datastore_id     = data.vsphere_datastore.datastore.id

    network_interface {
      network_id = data.vsphere_network.external_network.id
    }

    network_interface {
      network_id = data.vsphere_network.internal_network.id
    }

    disk {
      label            = "disk0"
      thin_provisioned = true
      size             = 50
    }

    clone {
      template_uuid = data.vsphere_virtual_machine.rocky9-template.id
#Linux_options are required section, while deploying Linux virtual machines
      customize {
          linux_options {
              host_name = "rocky91"
              domain = "net.local"
          }
          network_interface {
            ipv4_address = "10.10.1.5"
            ipv4_netmask = "23"
          }
          network_interface {
            #ipv4_address = "192.168.80.11"
            #ipv4_netmask = "24"
          }
          ipv4_gateway = "10.10.1.1"
      }

    }

    lifecycle {
       ignore_changes = [hv_mode, ept_rvi_mode]
    }
}

#Outup section will display vsphere_virtual_machine.ubu-testing Name and IP Address
output "VM_Name" {
  value = vsphere_virtual_machine.rocky9-vm.name
}

output "VM_IP_Address" {
  value = vsphere_virtual_machine.rocky9-vm.guest_ip_addresses
}

Debug Output

 ~ resource "vsphere_virtual_machine" "default" {
        id                                      = "421ae1e6-f4cf-6ab7-2ea7-3ec005a97055"
      ~ memory                                  = 6144 -> 8192

  ~ resource "vsphere_virtual_machine" "default" {
        id                                      = "421a661b-f5b6-619c-2e59-74175e5f562d"
        name                                    = "vm2"
      ~ num_cpus                                = 4 -> 6

Above two changes should not reboot the target VMs.

Panic Output

No response

Expected Behavior

Increasing CPU cores or memory size should take effect immediately without rebooting the target VM.

Actual Behavior

Increasing CPU core and memory size both cause the target VM reboot.

Steps to Reproduce

Create a RockyLinux 9.3 x86_64 template from base OS installation. Installing open-vm-tools from RockyLinux repo.
Provision a new VM from the template using above terraform config, which will enable hot add for CPU and memory.
Make a change to CPU, run terraform apply.
Make a change to memory, run terraform apply

Environment Details

No response

Screenshots

No response

References

No response

github-actions[bot] commented 6 months ago

Hello, sgaosdgr! 🖐

Thank you for submitting an issue for this provider. The issue will now enter into the issue lifecycle.

If you want to contribute to this project, please review the contributing guidelines and information on submitting pull requests.

sgaosdgr commented 6 months ago

Please note, this is NOT a duplicate to 1902 bug report. Currently, hot add CPU and memory support is broken.

tenthirtyam commented 4 months ago

I am unable to reproduce this issue using the following:

Terraform v1.8.5
Terraform Provider for VMware vSphere v2.8.1
VMware vSphere 8.0 Update 2

Tests Provided Below:

🟢 PASS: Enabling Hot Add for Both CPU and Memory (Reboot expected and observed.)

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform apply -auto-approve
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_network.network: Reading...
data.vsphere_datastore.datastore: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
      ~ cpu_hot_add_enabled                     = false -> true
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
      ~ memory_hot_add_enabled                  = false -> true
        name                                    = "windows-test"
      ~ num_cores_per_socket                    = 2 -> 1
        tags                                    = []
        # (70 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
vsphere_virtual_machine.vm: Modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 10s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 20s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 30s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 40s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 50s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 1m0s elapsed]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 1m10s elapsed]
vsphere_virtual_machine.vm: Modifications complete after 1m15s [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

🟢 PASS: Hot Add Memory (No reboot observed, as expected.)

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform plan               
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_datastore.datastore: Reading...
data.vsphere_network.network: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform plan
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_network.network: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_datastore.datastore: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
      ~ memory                                  = 4096 -> 8192
        name                                    = "windows-test"
        tags                                    = []
        # (72 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you
run "terraform apply" now.

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform apply -auto-approve
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_network.network: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_datastore.datastore: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
      ~ memory                                  = 4096 -> 8192
        name                                    = "windows-test"
        tags                                    = []
        # (72 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
vsphere_virtual_machine.vm: Modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 10s elapsed]
vsphere_virtual_machine.vm: Modifications complete after 10s [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

🟢 PASS: Hot Add CPU (No reboot observed, as expected.)

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] took 11.8s 
✦2 ➜ terraform plan               
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_datastore.datastore: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_network.network: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 1s [id=network-18085]
data.vsphere_datastore.datastore: Read complete after 1s [id=datastore-18076]
data.vsphere_compute_cluster.cluster: Read complete after 1s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 1s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
        name                                    = "windows-test"
      ~ num_cpus                                = 2 -> 4
        tags                                    = []
        # (72 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you
run "terraform apply" now.

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform apply -auto-approve
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_network.network: Reading...
data.vsphere_datastore.datastore: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
        name                                    = "windows-test"
      ~ num_cpus                                = 2 -> 4
        tags                                    = []
        # (72 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
vsphere_virtual_machine.vm: Modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 10s elapsed]
vsphere_virtual_machine.vm: Modifications complete after 10s [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

🟢 PASS: Hot Add CPU and Memory (No reboot observed, as expected.)

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] took 11.8s 
✦2 ➜ terraform plan               
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_datastore.datastore: Reading...
data.vsphere_network.network: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_datastore.datastore: Read complete after 1s [id=datastore-18076]
data.vsphere_compute_cluster.cluster: Read complete after 1s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_virtual_machine.template: Read complete after 1s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
      ~ memory                                  = 8192 -> 10240
        name                                    = "windows-test"
      ~ num_cpus                                = 4 -> 6
        tags                                    = []
        # (71 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you
run "terraform apply" now.

terrafom-examples-vmware/vsphere/vsphere-virtual-machine/clone-template-windows-guest-customization on  main [✘!?] 
✦2 ➜ terraform apply -auto-approve
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-3]
data.vsphere_network.network: Reading...
data.vsphere_datastore.datastore: Reading...
data.vsphere_compute_cluster.cluster: Reading...
data.vsphere_virtual_machine.template: Reading...
data.vsphere_network.network: Read complete after 0s [id=network-18085]
data.vsphere_compute_cluster.cluster: Read complete after 0s [id=domain-c18070]
data.vsphere_resource_pool.pool: Reading...
data.vsphere_datastore.datastore: Read complete after 0s [id=datastore-18076]
data.vsphere_virtual_machine.template: Read complete after 0s [id=421e4c56-9b9e-4d22-e171-922836680a14]
data.vsphere_resource_pool.pool: Read complete after 0s [id=resgroup-18071]
vsphere_virtual_machine.vm: Refreshing state... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "vm" {
        id                                      = "421e1ce0-9a8a-12f9-fb8a-fb8c724db879"
      ~ memory                                  = 8192 -> 10240
        name                                    = "windows-test"
      ~ num_cpus                                = 4 -> 6
        tags                                    = []
        # (71 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
vsphere_virtual_machine.vm: Modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]
vsphere_virtual_machine.vm: Still modifying... [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879, 10s elapsed]
vsphere_virtual_machine.vm: Modifications complete after 10s [id=421e1ce0-9a8a-12f9-fb8a-fb8c724db879]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Additionally, for the last test systeminfo was run before and after the increase for both the CPU and Memory as seen in the screenshot below.

Ryan Johnson Distinguished Engineer, VMware by Broadcom

github-actions[bot] commented 3 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-vsphere