Prioritize host interfaces for ipv4_address/ipv6_address/mac

[al20ov]

Hi, first of all I'd like to thank all the maintainers for this provider, it's awesome. Recently, I've been trying to automate the creation of a kubernetes cluster (k3s, RKE2, k0s and others) on Incus VMs and everything works well enough except for when I try to run terraform apply a second time after installing a kubernetes distribution:

Kubernetes components create a bunch of networks, bridges and so on and these addresses show up in the Incus webui as well as in the CLI. Only problem is, the ipv4_address and ipv6_address exported values from my incus_instance resources change from a private IPv4 on the same nework I run terraform from to a private IP only accessible from the networks created by Kubernetes.

Example:

Upon running a second terraform apply, the IPv4 changes

Changes to Outputs:
  ~ instance-ip = "10.127.0.41" -> "10.42.0.0"
null_resource.kubeconfig: Destroying... [id=5068605674949049145]
null_resource.kubeconfig: Destruction complete after 0s
null_resource.get-kubeconfig: Creating...
null_resource.get-kubeconfig: Provisioning with 'local-exec'...
null_resource.get-kubeconfig (local-exec): Executing: ["/bin/sh" "-c" "ssh -o 'StrictHostKeyChecking no' kube@10.42.0.0 sudo cat /etc/rancher/rke2/rke2.yaml > temp-kubeconfig"]
null_resource.get-kubeconfig: Still creating... [10s elapsed]
...
[Times out eventually because 10.42.0.0 is not reachable from my computer]

+-----------+---------+-----------------------+--------------------------------+-----------------+-----------+
|   NAME    |  STATE  |         IPV4          |              IPV6              |      TYPE       | SNAPSHOTS |
+-----------+---------+-----------------------+--------------------------------+-----------------+-----------+
| instance0 | RUNNING | 10.42.0.0 (flannel.1) | 2001::e91 (enp5s0) | VIRTUAL-MACHINE | 0         |
|           |         | 10.127.0.41 (enp5s0)  |                                |                 |           |
+-----------+---------+-----------------------+--------------------------------+-----------------+-----------+

In this example, the address I'm trying to reach instance0 from is 10.127.0.41 and that's what ipv4_address returns at first, but upon installing RKE2, ipv4_address returns 10.42.0.0 which is not accessible from my computer.

Now what I think is happening is something is grabbing the first address (index 0) from a list of IP addresses given by Incus agent and rolls with it. It's not a problem when the VM/container only has one address, but could become incorrect when it has more than one.

Unfortunately, it seems that that address is not always the one we're interested in. Maybe ipv4_addresses could be changed to a map of interface names to IPs? enp5s0 -> 10.127.0.41, flannel.1 -> 10.42.0.0, and so on?

[stgraber]

Did you try setting user.access_interface on your instance? The documentation mentions it as a way to specify what interface to look at for IPv4, IPv6 and MAC address.

So user.access_interface=enp5s0 should do the right thing for you.

Now in general, I think it'd be great to prioritize interfaces that we think came from the host. So for the default IPv4/IPv6/MAC, I think we should change the logic to do:

• Iterate over all interfaces
• Ignore loopback device
• Prioritize interfaces that have a host_name value
• Prioritize interfaces that have IPv4 or IPv6 set

[al20ov]

Thanks, I didn't know about that config key, I'm going to try it later today. As for the logic, I see that findIPv6Address already checks if the address is global. Should the additional checks be implemented there or maybe before that function gets called?

[stgraber]

I've reworked things quite a bit in the PR I opened a few minutes ago so that hopefully in 99% of cases, no manual tweaking should be needed.