gateway: fix changing IP addresses for same MAC during pxe install

[herzogf]

We run in the same problem mentioned here: https://dnsmasq-discuss.thekelleys.org.narkive.com/ZoFQNaGo/always-ignore-client-identifier#post4

During PXE-install the new node gets an IP address with infinite lease based on MAC address AND client-id (client identifier) sent by the debian installer. After installation and upon reboot the node gets a different IP address (not the IP address during install) because the cliend-id changes.

Sample output from our dummy dhcp_hook script:

add fa:2d:37:78:7f:cf 192.168.2.106 k8s-node-106 01:fa:2d:37:78:7f:cf
...
add fa:2d:37:78:7f:cf 192.168.2.107 k8s-node-107 ff:37:78:7f:cf:00:01:00:01:25:d4:61:4d:fa:2d:37:78:7f:cf

Same MAC address (column 2) but different client-id (last column).

Some 5 years after requesting a feature to ignore client-ids for all DHCP requests there's finally a commit in dnsmasq that adds a new option "dhcp-ignore-clid" that does exactly what we need: http://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=13a58f9590a7071ad4a8c8f7e4178c29a6b66be3

This feature should make it into dnsmasq 2.81 (current release is 2.80 , see http://thekelleys.org.uk/dnsmasq/CHANGELOG).

For a real fix we need to wait for dnsmasq 2.81 (or rather "the next release") and then add this option dhcp-ignore-clid. This issue stays open till this happens.

As a workaround we probably have to either

• set a fixed client-id during pxe install and somehow give a special handling for this in dnsmasq, either set a fixed ip-address for this which we ignore in our dhcp-hook (bad as we then never could have multiple nodes getting installed at the same time) or magically ignore exactly this client-id (doubt that this is possible in dnsmasq.conf)
• perhaps identify a pxe-install client-id in our dhcp-hook script (we see above that the "final" client-id after installation is much longer than the client-id during pxe-install) and then ignore it regarding ansible and in addition delete the lease in dnsmasq (see #54 )
• worst case: let our dhcp-hook script handle this pxe-install lease exactly like any other: it adds it to the ansible inventory, starts the ansible playbook and fails to connect. We then need a regular cleanup job which a) scans the dnsmasq leases for duplicate entries for the same MAC and delete the older one and b) cleans the ansible inventory file

[herzogf]

Short note for me: When doing the workaround where our dhcp-hook script is intelligent enough to recognize pxe-install leases we cannot just release the lease directly as we don't know when the lease isPXEClient not needed anymore (i.e. installation has ended). Two options come to my mind:

• either handle this with dnsmasq.conf: somehow recognize DHCP requests from pxe (see "dhcp-vendorclass" in dnsmasq options) and then specify a separate dhcp-range just for the pxe-install (e.g. 10 IP addresses for pxe with different hostnames and a lease-time of just 1 hour) -> then our dhcp-hook script could easily just check whether the hostname matches "k8s-node-.." and ignore anything else. The pxe dhcp-leases would just evaporate after an hour automatically. Some dnsmasq.conf like the following untested snipped could work:

  # set tag for PXE clients
  dhcp-vendorclass=set:PXEINSTALL,PXEClient
  # specify ip address range for pxe clients, low lease time
  dhcp-range=tag:PXEINSTALL,192.168.2.20,192.168.2.30,2h
  # specify ip adress range for provisioned k8s nodes (i.e. after pxe install + reboot) with infinite lease time)
  dhcp-range=tag:!PXEINSTALL,192.168.2.100,192.168.2.200,infinite
  # we could make this even better by providing 3 ranges: a small one for pxe clients, a rather large one with infinite lease time for k8s nodes and a third, small range for "other" dynamic clients, e.g. your admin's laptop

• the other option is to let ansible clean up the leases file after a k8s node is successfully provisioned, i.e. check the leases file for duplicat entries for the same MAC address and delete the older one

[herzogf]

Another option (perhaps the easiest as a workaround until dnsmasq is released with the new "ignore client-id" feature): just set the dhcp client-id for the installed debian system to the same id the pxe debian installer uses. See https://github.com/tsugliani/packer-vsphere-templates/blob/5bd06b553f4ee270aaa607c8508981cb52dfd807/scripts/Linux/Debian/preseed.cfg#L38 as an example (but the debian buster installer seems to send "01:" (with leading 0))