search

dmacvicar / terraform-provider-libvirt

Terraform provider to provision infrastructure with Linux's KVM using libvirt
Apache License 2.0
1.54k stars 457 forks source link

Waiting for lease finishes prematurely if domain has network interface WITHOUT IP addess assigned yet. #1047

Closed pstrzelczak closed 7 months ago

pstrzelczak commented 7 months ago

System Information

Linux distribution

Rocky 8.8

Terraform version

2023-11-06T10:41:59.199Z [INFO]  Terraform version: 1.6.2
2023-11-06T10:41:59.199Z [DEBUG] using github.com/hashicorp/go-tfe v1.36.0
2023-11-06T10:41:59.199Z [DEBUG] using github.com/hashicorp/hcl/v2 v2.19.1
2023-11-06T10:41:59.199Z [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2023-11-06T10:41:59.199Z [DEBUG] using github.com/zclconf/go-cty v1.14.1
2023-11-06T10:41:59.199Z [INFO]  Go runtime version: go1.21.1
2023-11-06T10:41:59.199Z [INFO]  CLI args: []string{"terraform", "--version"}
2023-11-06T10:41:59.200Z [DEBUG] Attempting to open CLI config file: /home/docker_user/.terraformrc
2023-11-06T10:41:59.200Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2023-11-06T10:41:59.200Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2023-11-06T10:41:59.200Z [DEBUG] ignoring non-existing provider search directory /home/docker_user/.terraform.d/plugins
2023-11-06T10:41:59.200Z [DEBUG] ignoring non-existing provider search directory /home/docker_user/.local/share/terraform/plugins
2023-11-06T10:41:59.200Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2023-11-06T10:41:59.200Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2023-11-06T10:41:59.200Z [INFO]  CLI command args: []string{"version", "--version"}
Terraform v1.6.2
on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.7.1
+ provider registry.terraform.io/hashicorp/null v3.2.1

Your version of Terraform is out of date! The latest version
is 1.6.3. You can update by downloading from https://www.terraform.io/downloads.html

Provider and libvirt versions

v0.7.1

Checklist

Description of Issue/Question

Setup

Any with wait_for_lease=true that later references network_interface.0.addresses.0 attributes:

output "rocky_hosts" {
  value = libvirt_domain.rocky-kvm.*.network_interface.0.addresses.0
}

Steps to Reproduce Issue

Run terraform apply.

The execution fails with:

│ Error: Invalid index
│ 
│   on ../modules/rocky-kvm-domains/main.tf line 82, in output "rocky_hosts":
│   82:   value = libvirt_domain.rocky-kvm.*.network_interface.0.addresses.0
│ 
│ The given key does not identify an element in this collection value: the
│ collection has no elements.

I reproduced this with TF_LOG=debug and found a procedure to wait for lease returns even if there are no IP addresses assigned:

2023-11-06 08:52:22 <DEBUG> --- 2023-11-06T07:52:22.792Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:22 [INFO] Domain ID: aad7a4a7-65e9-406c-ae69-d79bd2536dfc: timestamp=2023-11-06T07:52:22.792Z
2023-11-06T07:52:22.792Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:22 [DEBUG] Waiting for state to become: [all-addresses-obtained]: timestamp=2023-11-06T07:52:22.792Z
2023-11-06 08:52:27 <DEBUG> --- 2023-11-06T07:52:27.798Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] waiting for network address for iface=52:54:00:AA:7B:80: timestamp=2023-11-06T07:52:27.798Z
2023-11-06 08:52:27 <DEBUG> --- 2023-11-06T07:52:27.799Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] qemu-agent used to query interface info: timestamp=2023-11-06T07:52:27.799Z
2023-11-06 08:52:27 <DEBUG> --- 2023-11-06T07:52:27.800Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] Interfaces info obtained with libvirt API:
([]libvirt.DomainInterface) <nil>: timestamp=2023-11-06T07:52:27.800Z
2023-11-06T07:52:27.800Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] ifaces with addresses: []: timestamp=2023-11-06T07:52:27.800Z
2023-11-06T07:52:27.800Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] 52:54:00:AA:7B:80 doesn't have IP address(es) yet...: timestamp=2023-11-06T07:52:27.800Z
2023-11-06T07:52:27.800Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [DEBUG] IP address not found for iface=52:54:00:AA:7B:80: will try in a while: timestamp=2023-11-06T07:52:27.800Z
2023-11-06T07:52:27.800Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:27 [TRACE] Waiting 10s before next try: timestamp=2023-11-06T07:52:27.800Z
2023-11-06 08:52:31 <DEBUG> --- module.domains.libvirt_domain.centos-kvm[0]: Still creating... [10s elapsed]
2023-11-06 08:52:37 <DEBUG> --- 2023-11-06T07:52:37.809Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] waiting for network address for iface=52:54:00:AA:7B:80: timestamp=2023-11-06T07:52:37.809Z
2023-11-06 08:52:37 <DEBUG> --- 2023-11-06T07:52:37.810Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] qemu-agent used to query interface info: timestamp=2023-11-06T07:52:37.810Z
2023-11-06 08:52:37 <DEBUG> --- 2023-11-06T07:52:37.844Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] Interfaces info obtained with libvirt API:
([]libvirt.DomainInterface) (len=2 cap=2) {
    (libvirt.DomainInterface) {
        Name: (string) (len=2) "lo",
        Hwaddr: (libvirt.OptString) (len=1 cap=1) {
            (string) (len=17) "00:00:00:00:00:00"
        },
        Addrs: ([]libvirt.DomainIPAddr) (len=2 cap=2) {
            (libvirt.DomainIPAddr) {
                Type: (int32) 0,
                Addr: (string) (len=9) "127.0.0.1",
                Prefix: (uint32) 8
            },
            (libvirt.DomainIPAddr) {
                Type: (int32) 1,
                Addr: (string) (len=3) "::1",
                Prefix: (uint32) 128
            }
        }
    },
    (libvirt.DomainInterface) {
        Name: (string) (len=4) "ens3",
        Hwaddr: (libvirt.OptString) (len=1 cap=1) {
            (string) (len=17) "52:54:00:aa:7b:80"
        },
        Addrs: ([]libvirt.DomainIPAddr) <nil>
    }
}: timestamp=2023-11-06T07:52:37.844Z
2023-11-06T07:52:37.844Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] ifaces with addresses: [{Name:lo Hwaddr:[00:00:00:00:00:00] Addrs:[{Type:0 Addr:127.0.0.1 Prefix:8} {Type:1 Addr:::1 Prefix:128}]} {Name:ens3 Hwaddr:[52:54:00:aa:7b:80] Addrs:[]}]: timestamp=2023-11-06T07:52:37.844Z
2023-11-06T07:52:37.844Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] found IPs for MAC=52:54:00:AA:7B:80: []: timestamp=2023-11-06T07:52:37.844Z
2023-11-06 08:52:37 <DEBUG> --- 2023-11-06T07:52:37.844Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] all the 1 IP addresses obtained for the domain: timestamp=2023-11-06T07:52:37.844Z
2023-11-06T07:52:37.845Z [INFO]  provider.terraform-provider-libvirt_v0.7.1: 2023/11/06 07:52:37 [DEBUG] wait-for-leases was successful: timestamp=2023-11-06T07:52:37.844Z

According to my investigation this is caused by https://github.com/dmacvicar/terraform-provider-libvirt/commit/484dec2e02a1a2c624d266bdc905235e9786d957 because before that commit was merged domainGetIfacesInfo was using qemuAgentGetInterfacesInfowhich did not return network interface that had no IPv4 address assigned:

                        if len(libVirtIface.Addrs) > 0 && (ipv4Assigned || !wait4ipv4) {
                                interfaces = append(interfaces, libVirtIface)
                        }

With this part removed, already existing code introduced long time ago in https://github.com/dmacvicar/terraform-provider-libvirt/commit/c7cc7221d46e8fa7ee7d64c7eab64f5515980fc1 makes waiting for lease pass even though there is no IP address assigned:

        for _, ifaceWithAddr := range ifacesWithAddr {
                if len(ifaceWithAddr.Hwaddr) > 0 && (mac == strings.ToUpper(ifaceWithAddr.Hwaddr[0])) {
                        log.Printf("[DEBUG] found IPs for MAC=%+v: %+v\n", mac, ifaceWithAddr.Addrs)
                        return true, false, nil
                }
        }

From my analysis all versions starting with 0.6.9-pre1 are affected.

Additional information:

Do you have SELinux or Apparmor/Firewall enabled? Some special configuration? Not sure but I think it is irrelevant Have you tried to reproduce the issue without them enabled? No