self.network_interface[0].addresses is empty - terraform does not refresh state

[bestrocker221]

Linux distribution

Manjaro 5.13.19

Terraform version

Terraform v1.1.2
on linux_amd64  
+ provider registry.terraform.io/dmacvicar/libvirt v0.6.12

Description of Issue/Question

Note: Using user session with bridged interfaces. qemu_agent = true wait_for_lease = true

When creating a single domain (VM) libvirt successfully gets the IP address of the machine.

I use self.network_interface.0.addresses[0] within the domain definition to get it.

When I create multiple domains (VMs) within the same domain block, with the use of for_each, virtual machines are created and spun up, but terraform fails to complete because:

Error: Invalid index
...
  self.network_interface[0].addresses is empty list of string
...

After the fail, I run a terraform show and I see that:

network_interface {                                                                                                                                                                                                                                                                                      
        addresses      = []
        ...

Adresses is actually empty.

I run a terraform refresh and now the IP shows up.

It is not a problem of DHCP, because the IP is released immediately to the VM, since the same setup is working when creating a single VM.

In some way, terraform does not wait/get the IP when multiple machines are created.

Any idea or suggestions I might try?

Thank you

[gpzj]

Same issue here, also getting no results for addresses via "terraform show" until I do a refresh. Cannot run post-provisioner due to this.

[gpzj]

Ah - I think this is the solution: wait_for_lease = true

(full block exampel) resource "libvirt_domain" "ubuntu" { network_interface { network_name = "default" wait_for_lease = true } }

[bestrocker221]

I solved the problem by switching to DNS resolutions of the machines, but this issue persists. As stated initially, I always set the wait_for_lease=true already

[andyschmidt]

I experience the same problem when using count > 1

network_interface { network_name = "default" hostname = "${var.hostname}-${count.index}" wait_for_lease = true }

and

connection { type = "ssh" host = self.network_interface.0.addresses[0] user = "root" password = "root" }

only the first instance can be provisioned with the "file" provisioner. the other domains time out with this error message:

│ Error: couldn't retrieve IP address of domain id: 216fffa8-1e7a-4559-88f6-20cd6e0ac2c3. Please check following: │ 1) is the domain running proplerly? │ 2) has the network interface an IP address? │ 3) Networking issues on your libvirt setup? │ 4) is DHCP enabled on this Domain's network? │ 5) if you use bridge network, the domain should have the pkg qemu-agent installed │ IMPORTANT: This error is not a terraform libvirt-provider error, but an error caused by your KVM/libvirt infrastructure configuration/setup │ timeout while waiting for state to become 'all-addresses-obtained' (last state: 'waiting-addresses', timeout: 5m0s) │ │ with libvirt_domain.domain-terraform[1], │ on template-vm.tf line 50, in resource "libvirt_domain" "domain-terraform": │ 50: resource "libvirt_domain" "domain-terraform" { │

[gpzj]

Ah shoot, sorry @bestrocker221 - I see now that you did have the wait in your original config.

I am on slightly newer versions of terraform and the libvirt provider FWIW.

Terraform v1.1.4 on linux_amd64

• provider registry.terraform.io/dmacvicar/libvirt v0.6.13 Pop!_OS 21.10

[dmacvicar]

I tried to reproduce this with the following, without success:

terraform {
 required_version = ">= 0.13"
  required_providers {
    libvirt = {
      source  = "dmacvicar/libvirt"
      version = "0.6.2"
    }
  }
}

provider "libvirt" {
  uri = "qemu:///system"
}

resource "libvirt_volume" "os_image" {
  name   = "os_image"
  source = "http://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.3/images/openSUSE-Leap-15.3.x86_64-NoCloud.qcow2"
}

resource "libvirt_volume" "volume" {
  name           = "volume-${count.index}"
  base_volume_id = libvirt_volume.os_image.id
  count          = 2
}

resource "libvirt_domain" "domain" {
  name = "domain-${count.index}"

  disk {
    volume_id = element(libvirt_volume.volume.*.id, count.index)
  }

  network_interface {
    network_name   = "default"
    wait_for_lease = true
  }

  count = 2
}

output "ips" {
  value = libvirt_domain.domain.*.network_interface.0.addresses
}

note that my ~/.terraformrc overrides the provider with my local build:

$ cat ~/.terraformrc
provider_installation {
  dev_overrides {
    "dmacvicar/libvirt" = "/space/git/terraform-provider-libvirt"
  }
  direct {}
}

After terraform apply:

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Outputs:

ips = [
  tolist([
    "192.168.122.167",
  ]),
  tolist([
    "192.168.122.190",
  ]),
]

So this is terraform 1.1.2 and provider from master.

[bestrocker221]

Hi @dmacvicar ! I noticed that you use the system session, and can it be that that's why you don't encounter any problems?

I mentioned I use the user session (and bridged int), and that comes with a different approach to IP resolution right? since the user session does not have access to the DHCP lease list.. and instead uses the qemu-agent. (for what I understood)

I also tried your exact configuration, with the only change to my bridged interfaces and adding qemu-agent=true, and the result is that only the IPv6 are shown, not the IPv4