set custom cpu model - to enable kvm live migration

tbals commented 1 year ago

System Information

Linux distribution

Ubuntu 20.04.6 LTS

Terraform version

Terraform v1.4.6

Provider and libvirt versions

dmacvicar/libvirt v0.7.1

Checklist

[ x] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?
- [ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
- [ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
- [ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

Description of Issue/Question

Setup

(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive information)

terraform {
  required_version = ">= 1.0.1"
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
      version = "0.7.1"
    }
  }
}

provider "libvirt" {
  uri = "qemu+ssh://terraform@${var.qemu_server}/system"
}

resource "libvirt_domain" "ubuntu-vm" {
  name = "${var.vm_id}__${var.hostname}"
  memory = var.ram
  vcpu   = var.cpu
  autostart = var.autostart

  cpu {
    mode = "host-passthrough"
  }

  disk {
    volume_id = libvirt_volume.ubuntu-vol.id
  }

  boot_device {
    dev = [ "hd", "network"]
  }

  dynamic "network_interface" {
    for_each = var.network
    content {
      bridge = network_interface.value
    }
  }

  cloudinit = libvirt_cloudinit_disk.cloud-init.id

  console {
    target_type = "serial"
    type        = "pty"
    target_port = "0"
  }
  console {
    target_type = "virtio"
    type        = "pty"
    target_port = "1"
  }
}

resource "libvirt_volume" "ubuntu-base" {
  name = "ubuntu-${var.dist}-${var.hostname}-base.qcow2"
  pool = "images"
  source = "https://ftp.domain.tld/src/${var.dist}-server-cloudimg-amd64.img"
  format = "qcow2"
}

resource "libvirt_volume" "ubuntu-vol" {
  name = "${var.hostname}.qcow2"
  pool = "images"
  base_volume_id = libvirt_volume.ubuntu-base.id
  size = var.vol_size
}

data "template_file" "user_data" {
  template = file("${path.module}/../user-data.yml")
  vars = {
    hostname = var.hostname
    fqdn = "${var.hostname}.${var.domain}"
    chroot = var.vm_id
    swap = var.swap_size
  }
}

data "template_file" "network_config" {
  template = file("${path.module}/../network_config.yml")
  vars = {
    ip4 = var.ip4
    gw4 = var.gw4
  }
}

resource "libvirt_cloudinit_disk" "cloud-init" {
  name = "cloud-init-${var.hostname}.iso"
  pool = "cloud-init"
  user_data = data.template_file.user_data.rendered
  network_config = data.template_file.network_config.rendered
}

Additional information:

For enable libvirt live migration I need to set a specific CPU model in XML definition file.

Live migration dont work with "Host passthrough", you need to set a Named model. In my case it looks like:

  <cpu mode='custom' match='exact' check='partial'>
    <model fallback='allow'>Haswell-noTSX-IBRS</model>
  </cpu>

Also a specific os machine type is needed for live migration:

  <os>
    <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
       :
  </os>

I could not find any documentation for setting a CPU model and nothing for set an OS machine type. Is there any way to do this?

Documentation: https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html

tbals commented 1 year ago

more docs about this:

https://libvirt.org/formatdomain.html#cpu-model-and-topology https://libvirt.org/formatdomain.html#bios-bootloader https://libvirt.org/formatcaps.html

tbals commented 1 year ago

without setting the <cpu> feature in main.tf, the deployed VM looks like:

  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>qemu64</model>
  </cpu>

But with this CPU setting a live migration is not possible:

# virsh migrate --live u22-migtest qemu+ssh://dest-system/system
error: operation failed: guest CPU doesn't match specification: extra features: vmx

dmacvicar / terraform-provider-libvirt