Vsphere static IP allocation sequence

[haisum]

I am provisioning RKE cluster using vsphere cloud provider. I have a pool of IP addresses/hostnames defined in a network profile from which cluster nodes should be provisioned. I am passing vapp properties and using a cloud-init script to set up network interface. Can you please help me in figuring out how can I achieve following:

I expect to have cluster on vsphere such as:

VM Name	Hostname	IP
phlaskubtstc2n1	phlaskubtstc2n1	10.88.197.160
phlaskubtstc2n2	phlaskubtstc2n2	10.88.197.161
phlaskubtstc2n3	phlaskubtstc2n3	10.88.197.162

But what I end up with is random allocation of ips like this:

VM Name	Hostname	IP
phlaskubtstc2n1	phlaskubtstc2n1	10.88.197.161
phlaskubtstc2n2	phlaskubtstc2n2	10.88.197.162
phlaskubtstc2n3	phlaskubtstc2n3	10.88.197.160

Here's what my node template looks like with terraform:

resource "rancher2_node_template" "tools-cluster" {
  name                = "tools-cluster"
  description         = "Tools cluster node template"
  cloud_credential_id = rancher2_cloud_credential.tools-cluster.id
  vsphere_config {
    cloud_config = templatefile("${path.module}/files/cloud-config.tpl.yml", {...})
   ...
    vapp_ip_allocation_policy = "fixedAllocated"
    vapp_ip_protocol          = "IPv4"
    vapp_transport            = "com.vmware.guestInfo"
    vapp_property = [
      "guestinfo.interface.0.ip.0.address=ip:${var.tools_cluster_infra.network}",
      "guestinfo.interface.0.ip.0.netmask=$${netmask:${var.tools_cluster_infra.network}}",
      "guestinfo.interface.0.route.0.gateway=$${gateway:${var.tools_cluster_infra.network}}",
      "guestinfo.dns.servers=$${dns:${var.tools_cluster_infra.network}}",
    ]
  }
}

resource "rancher2_node_pool" "tools-cluster" {
  cluster_id       = rancher2_cluster.tools-cluster.id
  name             = "tools-cluster"
  hostname_prefix  = var.tools_cluster_infra.vm_name_prefix
  node_template_id = rancher2_node_template.tools-cluster.id
  quantity         = 3
  control_plane    = true
  etcd             = true
  worker           = true
}

Here's relevant section from cloud-config file:

#cloud-config
write_files:
  - path: /root/cloud-init.sh
    permissions: "0755"
    content: |
        #!/usr/bin/env bash
        vmtoolsd --cmd 'info-get guestinfo.ovfEnv' > /root/ovfenv
        IPAddress=$(sed -n 's/.*Property oe:key="guestinfo.interface.0.ip.0.address" oe:value="\([^"]*\).*/\1/p' /root/ovfenv)
        SubnetMask=$(sed -n 's/.*Property oe:key="guestinfo.interface.0.ip.0.netmask" oe:value="\([^"]*\).*/\1/p' /root/ovfenv)
        Gateway=$(sed -n 's/.*Property oe:key="guestinfo.interface.0.route.0.gateway" oe:value="\([^"]*\).*/\1/p' /root/ovfenv)
        DNS=$(sed -n 's/.*Property oe:key="guestinfo.dns.servers" oe:value="\([^"]*\).*/\1/p' /root/ovfenv)
        nmcli con delete eth0
        nmcli con add con-name "static-eth0" ifname eth0 type ethernet ip4 "$IPAddress"/24 gw4 "$Gateway" ipv4.dns "$DNS"
        systemctl restart network
runcmd:
  - /root/cloud-init.sh

Network profile looks like this:

Here, the problem is with vmtoolsd --cmd 'info-get guestinfo.ovfEnv' in cloud-config is getting random ips from range of ips defined in network profile. So I end up with something like this on vsphere:

and on rancher UI:

Note in above that IP for phlaskubtstc2n3 is displayed as .160 but DNS server has phlaskubtstc2n3 = 10.88.197.162 entry so if someone was to do ssh phlaskubtstc2n3 they would end up at node which rancher and vsphere UI displays as phlaskubtstc2n2. This is cause of confusion for end users. The work around is to have one node template per node and pass static ips/hostnames via those templates but it gets really ugly with high number of nodes.

I would really appreciate if someone can help me with coming up with a solution which syncs ip/hostname on rancher and vSphere. One possible solution is to have rancher pass name of VM via guestinfo to guest OS. That way guest would know it's chosen to be host1 so should allocate IP1 to its' interface, but I couldn't find any way to do that.

What kind of request is this (question/bug/enhancement/feature request):

Question

Other details that may be helpful:

Environment information

• Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): v2.4.2
• Installation option (single install/HA): HA

Cluster information

• Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Vsphere
• Machine type (cloud/VM/metal) and specifications (CPU/memory): VM
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

[haisum]

We can pass MachineName decided by Rancher in vappProperties as guestinfo.machine.name to resolve this. In cloud-init one can fetch this info and set hostname/IP according to that. It's similar to approach defined here: https://pubs.vmware.com/esx254/admin/wwhelp/wwhimpl/common/html/wwhelp.htm?context=admin&file=esx25admin_vms.4.7.html . I can come up with a pull request if someone from Rancher team gives a go ahead.

[stale[bot]]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

[marvinlnnx]

this is because you have enabled DHCP in your network profile. that's why each time your vm getting different IP address.