dmacvicar / terraform-provider-libvirt

Terraform provider to provision infrastructure with Linux's KVM using libvirt
Apache License 2.0
1.58k stars 456 forks source link

[Packer X Terraform] Slot issue with network interfaces #1084

Open luclis opened 5 months ago

luclis commented 5 months ago

System Information

Linux distribution

Ubuntu 22.04

Terraform version

terraform -v
Terraform v1.8.2 on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.7.6

libvirtd --version
libvirtd (libvirt) 8.0.0

Checklist

Description of Issue/Question

When packaging a new vm with Packer over QEMU, the network interface use the ens4 name. When rebuilding it with Terraform, the network interface get the ens3 slot.

Setup

terraform {
  required_version = ">= 1.6.6"
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
      version = "0.7.6"
    }
}
provider "libvirt" {
  # Configuration du fournisseur libvirt
  uri = "qemu:///system"
}

resource "libvirt_pool" "default" {
    name = "Terraformed"
    type = "dir"
    path = "/srv/VMs"
}

resource "libvirt_network" "default" {
    name = "Nated"
    mode = "nat"
    domain = "lab.luclis.fr"
    addresses = ["10.0.0.0/24"]
    dns {
      enabled = "true"
      local_only = "false"
    }
}

// variables that can be overriden
variable "hostname" { default = "debian-nginx" }
variable "OS" { default = "Debian" }
variable "OSversion" { default = "12.04"}
variable "domain" { default = "lab.luclis.fr" }
variable "ip_type" { default = "dhcp" }
variable "memoryMB" { default = 1024*2 }
variable "cpu" { default = 2 }

resource "libvirt_volume" "nginx" {
  name = "${var.OS}"
  pool = libvirt_pool.default.name
  source = "/srv/outputs/${var.OS}-${var.OSversion}/${var.OS}-${var.OSversion}"
  format = "qcow2"
}

// Create the machine
resource "libvirt_domain" "nginx" {
  # domain name in libvirt, not hostname
  name = var.hostname
  memory = var.memoryMB
  vcpu = var.cpu
  qemu_agent = "true"
  disk {
    volume_id = libvirt_volume.nginx.id
    scsi = "true"
  }
  network_interface {
    network_id = libvirt_network.default.id
 #   wait_for_lease = "true"
  }

  graphics {
    type = "spice"
    listen_type = "address"
    autoport = "true"
  }
}

output "ips" {
  value = libvirt_domain.nginx.*.network_interface.0.addresses
}

Steps to Reproduce Issue

  1. Make a build with Packer
    
    build {
    sources = [
    "source.qemu.Debian"
    ]
    }
    packer {
    required_plugins {
    qemu = {
      version = "~> 1"
      source  = "github.com/hashicorp/qemu"
    }
    }
    }

variable "name" { sensitive = false default = "Debian-12.04" description = "The name of the firt VM created." }

variable "params" { sensitive = false default = { disk = "40960" format = "qcow2" ram = "4096" cpu = "2" } }

variable "accounts" { sensitive = true default = { username = "ansible" password = "ansible" } description = "The account of the first user." }

variable "iso" { default = { checksum = "file:https://cdimage.debian.org/mirror/cdimage/archive/12.4.0/amd64/iso-dvd/SHA256SUMS" url = "https://cdimage.debian.org/mirror/cdimage/archive/12.4.0/amd64/iso-dvd/debian-12.4.0-amd64-DVD-1.iso" } description = "The url of the iso." }

variable "guest_additions_iso" { default = { url = "https://download.virtualbox.org/virtualbox/6.1.50/VBoxGuestAdditions_6.1.50.iso" checksum = "file:https://download.virtualbox.org/virtualbox/{{.Version}}/SHA256SUMS" # Ne marche pas. Il prends pas le file on dirait. } description = "The url of the iso." }

variable "Port" { default = { min = 8081 max = 8081 } description = "The http server port. The value at min and max means only 1 port open." }

source qemu "Debian" { iso_url = var.iso.url iso_checksum = var.iso.checksum ssh_username= var.accounts.username ssh_password= var.accounts.password ssh_timeout= "30m" accelerator = "kvm" vm_name = var.name net_device = "virtio-net-pci" disk_interface = "virtio-scsi" disk_size = var.params.disk format = var.params.format memory = var.params.ram cpus = var.params.cpu

headless = true disk_compression = true

output_directory = "/srv/outputs/${var.name}" http_directory = "../../http" http_port_min = var.Port.min http_port_max = var.Port.max shutdown_command = "echo ${var.accounts.password} | sudo -S shutdown -P now" boot_command = [ "auto console-keymaps-at/keymap=fr url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg", ] }

2. Apply it with the terraform previously write
3. Connect to QEMU and see that the name of the interface isn't right. 
4. run `lspci` : 
```bash
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 [...]
00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:04.0 SCSI storage controller: Red hat, Inc. Virtio SCSI

If you look at the packer build, the Ethernet and SCSI are inverted.

How to fix :

  1. Find a way to manually choose which device goes where
  2. In accord with the Packer Team of QEMU, force the same address for both ?

Workaround

  1. Maybe a xsl file can made the change. I've try but have always an error with the xslt.
  2. In the packer boot command line, add net.ifnames=0 biosdevname=0 to set the interface name to eth0.

Thanks for your work, he's already amazing.

scabala commented 3 weeks ago

I must say I do not entirely know if this is more a libvirt bug, terraform-provider-libvirt bug or Packer bug. Even though that libvirt uses QEMU under the hood, I think it is safe to expect difference in usage by PAcker and libvirt.

What I can advise immediately is to try to replicate this bug with pure libvirt using virt-install/virsh. IF it happens, that's likely a libvirt bug rather than this provider one.