ESXi Guest isn't getting its Static IP and I can't see my mistake -- halp?

[bish0polis]

Hey guys,

I posted on the wiki project, and I think it was wrong. I'll kill it and I hope this is better.

So, Bad Things Happened [tm], and I've lost my templates, and I can not instantiate static-IP CentOS7 guests onto my ESXi host using tf-provider-ESXi . And I'm beginning to doubt I ever could !! :-D I'm hoping someone is in a similar spot and got through it. Stack-overflow has been its usual utility for nichier topics.

1. I have cloud-init (from the distro) and the guestinfo stuff added into the template
2. I'm instantiating something successfully, finally
3. the client is getting the meta-data
4. cloud-init isn't messing with the ifconfig how I'd like
5. and no static IP; it's doing DHCP

I'm running with packages, so here are the package versions for tools on the (gitlab-runner) host:

golang-github-terraform-provider-esxi-1.9.1-0.1.x86_64
terraform-0.12.31-1.el7.harbottle.x86_64
golang-github-terraform-provider-dns-3.2.1-1.x86_64
VMware ovftool 4.4.0 (build-15722219)

And the package versions on the generated VM:

centos-release-7-9.2009.1.el7.centos.x86_64
cloud-init-19.4-7.el7.centos.6.x86_64
cloud-init-vmware-guestinfo-1.4.1-1.noarch

cloud-init-vmware-guestinfo-1.4.1-1.noarch.rpm:

/etc/cloud/cloud.cfg.d/99-DataSourceVMwareGuestInfo.cfg
/usr/bin/dscheck_VMwareGuestInfo
/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceVMwareGuestInfo.py
/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceVMwareGuestInfo.pyc
/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceVMwareGuestInfo.pyo
/usr/share/doc/cloud-init-vmware-guestinfo-1.4.1
/usr/share/doc/cloud-init-vmware-guestinfo-1.4.1/README.md

Kickstart packages to build the template in packer-esxi:

%packages
@^minimal
@core
yum-utils
yum-plugin-downloadonly

chef < 15
wget

# cloud-init
cloud-init
cloud-init-vmware-guestinfo

tcpdump
net-tools
iptables

bzip2
ntp
yum
-avahi
-aic94xx-firmware
-alsa-firmware
-ivtv-firmware
-iwl*firmware
-postfix
-yum-langpacks

%end

Terraform plan:

-/+ destroy and then create replacement
Terraform will perform the following actions:
  # esxi_guest.mylan["reflect"] must be replaced
-/+ resource "esxi_guest" "mylan" {
        boot_disk_size         = "20"
        boot_disk_type         = "thin"
      ~ clone_from_vm          = "CentOS-7.9-x86_64-20220504230543" -> "CentOS-7.9-x86_64-20220508165133" # forces replacement
        disk_store             = "cloud2-ssd-3t-1"
        guest_name             = "reflect"
      ~ guest_shutdown_timeout = 20 -> (known after apply)
      ~ guest_startup_timeout  = 120 -> (known after apply)
        guestinfo              = {
            "metadata"          = "bmV0d29yazoKICB2ZXJzaW9[...]bGF0eXB1cy5iYy5jYQo="
            "metadata.encoding" = "base64"
        }
      ~ guestos                = "centos-64" -> (known after apply)
      ~ id                     = "244" -> (known after apply)
      ~ ip_address             = "10.1.1.109" -> (known after apply)
        memsize                = "2048"
      + notes                  = (known after apply)
        numvcpus               = "2"
      ~ ovf_properties_timer   = 0 -> (known after apply)
        power                  = "on"
        resource_pool_name     = "/"
      ~ virthwver              = "9" -> (known after apply)
      ~ network_interfaces {
          + mac_address     = (known after apply)
            nic_type        = "vmxnet3"
            virtual_network = "VM Network"
        }
    }

The meta-data appears to come in cleanly, and shows guest-info:

[root@reflect ~]# vmware-rpctool "info-get guestinfo.metadata" | base64 -d
network:
  version: 1
  config:
    - type: physical
      name: eth0
      subnets:
        - type: static
          address: 10.1.1.249/24
          gateway: 10.1.1.1
          dns_nameservers:
            - 10.1.1.1
          dns_search:
            - mylan.my.net

However, the host retains its DHCP config, despite meddling by cloud-init to ideally out-wit networkmangler:

[root@reflect ~]# more /etc/sysconfig/network-scripts/ifcfg-eth0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=dhcp
DEVICE=eth0
HWADDR=00:0c:29:17:18:a1
ONBOOT=yes
STARTMODE=auto
TYPE=Ethernet
USERCTL=no

net result:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:17:18:a1 brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.109/24 brd 10.1.1.255 scope global noprefixroute dynamic eth0

I've trimmed the listings a bit for space, but I'll expand them if we need to. Everything's building on CI now, so I can roll new images with changes with minimal fuss.

But I'm just not seeing why the meta-data passed into ESXi and onto the VM isn't being used to configure the NIC as hoped. Anything sticking out as missing?

[bish0polis]

Please, any idea you have. I'm really jammed-up here, and I'll try anything plausible.

[josenk]

This wiki is for the esxi provider. Your problem is related to providing.

[bish0polis]

Okay, fair point.

But I can't be 100.0% sure that the issue I'm seeing isn't the provider acting up -- and, also, I'm thinking if there are any others out there doing C7 on ESXi like I am, that they'll notice the obvious thing I'm not seeing.

I'm going back to the documentation and doign that abhorrent 'wget|sh' installation for the cloud-init add-on, in the hopes of learning there's a fixable issue with the packaged form.

Are we content that as long as the meta-info gets to ESXi, that terraform-esxi's job is done? Can we strike it off the list of suspects if "info-get guestinfo.metadata" gets us a good response, or is there a more conclusive test?