[FEATURE] Statically bind the IP address for VMs during creation

[starbops]

Is your feature request related to a problem? Please describe.

We've introduced the managed DHCP feature by implementing vm-dhcp-controller (an experimental add-on at the moment). It could be used as an internal DHCP server for VM Networks on Harvester clusters. However, the current integration with Harvester itself is loose. For example, there is no way to assign an IP address for the VM during VM creation: vm-dhcp-controller will monitor the VMI objects and dynamically create the vmnetcfg (VirtualMachineNetworkConfig) objects, which means the allocated IP address from the pool will also be dynamically chosen. Unless the user creates the vmnetcfg object (with IP and MAC addresses provided) before creating the VM (and the MAC address must be configured during the VM creation for the DHCP static lease to work).

We need to provide a way for users to configure the designated IP address(es) for the VMs during creation.

Describe the solution you'd like

As @ibrokethecloud suggested, we could add the relevant annotations in the VM objects' metadata, holding the essential information for vm-dhcp-controller to generate the corresponding vmnetcfg objects. That way, we could support the major use cases with the same implementation:

1. Create VMs with statically bound IP addresses
2. Create VMs without specifying the desired IP addresses (will be dynamically allocated from the pool)
3. Create VMs opt-out managed DHCP

P.S. The current VMI-based reconcile loop only supports the second one.

Describe alternatives you've considered

N/A

Additional context

https://github.com/harvester/harvester/pull/4960#discussion_r1470404662

[harvesterhci-io-github-bot]

GUI issue created #6087.