no IP address assigned to new cloned 18.04 template

[j4y4r]

Hello,

i have an issue when deploying ubuntu18.04 Preview on an esxi version 6.0.

open-vm-tools version: 2:10.2.0-3ubuntu2

After creating a new VM from a Template with pyvmomi, the machine does not get an IP address assigned, also in the VM properties->Network the options "Connect" and "Connect at power on" are not checked.

The file /etc/network/interfaces still exists on the system, but there was nothing written into the file.

I also tried to create a new Vm from a Template via the vSphere Client with the same result.

Ubuntu changed their default Networking to netplan with 17.10 https://lists.ubuntu.com/archives/ubuntu-devel-announce/2017-June/001215.html.

If you need any further infomation, logfiles, etc. please let me know.

[mrlesmithjr]

Just adding my .02 here for reference. I am using vCenter 6.7 to start with. However, my scenario might be a bit different so let me explain.

The below involves Ubuntu 18.04 server w/out cloud-init.

I am building OVA images using Packer using the vsphere-iso builder. As part of the process I am having Packer remove the network interfaces as we may need to define different types of network adapter during provisioning. This OVA is uploaded to vCenter using Ansible as a new VM, remaining powered off. Then this VM is converted to a template via Ansible as well. When provisioning a VM the network adapter type is defined in which case the VM is spun up appropriately with the defined adapter type. The OS sees the adapter but the customization is failing during the netplan configuration.

Update After following the recommendation above which lead to this KB https://kb.vmware.com/s/article/56409, I still experienced the same issue. However, switching from a vmxnet2 adapter type to a vmxnet3, the issue is not experienced. Is this a known thing with vmxnet2 adapters by chance?

Update After looking at the VMware compatibility guide, it appears that vmxnet2 is not supported on Ubuntu 18.04 but was on 16.04.

[hangrao]

Update After looking at the VMware compatibility guide, it appears that vmxnet2 is not supported on Ubuntu 18.04 but was on 16.04.

Right, vmxnet2 has been superseded by vmxnet3, please use vmxnet3 or other supported nics.

[spaced]

had a similar issue using centos 7 1810 image. Problem found on the guest side: Missing perl package crashes the scripts for the customization guest pkgs. You may also have look in /var/log/vmware-incs/* logs after provisioning.

[PengpengSun]

Hi @spaced ,

perl package is necessary for guest customization to work, please include 'perl' at OS installation or install 'perl' package before do guest customization.

[rthimons85]

On ubuntu 18.04 cloud images, I found that /etc/netplan/50-cloud-init.yaml was conflicting with the vmware customizations. Disabling it by deleting 50-cloud-init.yaml and creating a file named /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg and putting in

network: 
  config: disabled

fixed the issue, and my customizations completed properly.

[PengpengSun]

Thanks @rthimons85, Ubuntu 18.04 cloud image should have cloud-init installed according to the /etc/netplan/50-cloud-init.yaml file exits. For Ubuntu 18.04 Live server image, it has cloud-init installed too. There is a KB article on disabling cloud-init if using perl based VMware customization.

[AndrewSav]

This is now closed as invalid over at LaunchPad Ubuntu.

[mylesagray]

Posted a blog on how to use cloud-init for full setup on vSphere: https://blah.cloud/infrastructure/using-cloud-init-for-vm-templating-on-vsphere/

This took me days to figure out, but it hinged on a few things (covered in the Troubleshooting section of the blog) - but essentially, do NOT change the open-vm-tools system unit file if you want to use cloud-init.

Also, it seemed to matter that in the OVA spec, a hostname was not set for the initial template VM, and so it used the default.

Clearing /etc/machine-id is also essential for DHCP leases to not conflict as netplan uses this instead of MAC address for uniqueness.

[dasmokedog]

If you are running VIO (VMware Integrated Openstack) and smashing your head against the wall trying to get cloud-init to identify the datasource, like me, then you will need to ensure vmtools (or open-vm-tools) runs and registers the IP with SpoofGuard. Until this happens all traffic will be blocked. The quickest way is to arm cloud-init to run on 2nd boot.

[vmwarelab]

@dasmokedog how do you configure cloud-init to run on 2nd boot instead of first boot?

[dasmokedog]

@dasmokedog how do you configure cloud-init to run on 2nd boot instead of first boot?

@vmwarelab Initially I approached it by adding /etc/cloud/cloud-init.disabled at the end of the Packer run before it creates the image. I also included code in rc.local that would simply write and check for a file. If the file exists then the server has been rebooted and it would delete the /etc/cloud/cloud-init.disabled file and execute cloud-init clean to do a clean run. This had mixed results as rc.local is executed at the end of each run level. So I scrapped that and decided the to create a simple service. The service does a few things it will prep the server then run cloud-init clean, write a file to the /var/lib/cloud dir indicating it was ran but not validated. When the server comes back up it checks for that file then if it finds it the script will run validation checks to ensure that cloud-init was successful (checks the host name is properly set to the hash of its IP, curls metadata and queries AD for a domain account) if any fail it will re-execute cloud-init clean. Once the server passes validation it writes a file to the same location indicating a successful run.

I admit it is a tad heavy handed but it has been the only way I have been able to get 100% success rate for ubuntu 18.04 on vIO. The problems imo come down to timing of when cloud-init runs and when the system is actually in a proper state for it to be ran, and a service seemed the logical route.

Attached is a text file that is the contents of the script and the unit file for the service. Its nothing special except I no longer get called because a users new instance failed to provision ;-)

cloud-init-fix.txt

[vmwarelab]

thanks for sharing @dasmokedog i was hoping for something easier.. its never easy when it comes to cloud-init and i am hoping i never have to use it on vSphere. its just ugly, not consistent results . meaning not always successful.

In my case i m testing it with vRA 8 and the only way currently working is if i use DHCP without any customization where the machine doesn't reboot. if i try to use a static IP assignment , the VM's NIC would be in a disconnected state when the VM is powered on and until customization is completed and the machine rebooted, the NIC then switch to a connected state . of course in that time Cloud-Init already tried running when there was no network and also gets interrupted by the customization reboot

still for having it work with DHCP the template had to be configured, like for example in my case :

1. I used this kb https://kb.vmware.com/s/article/56409
2. run sudo dpkg-reconfigure cloud-init and only select OVF only.
3. run sudo cloud-init clean --logs before shutting down the VM and turning it into a template.

In my case its a hit or mess still, for example if my deployment has two VMs being provisioned from the same image i prepared but with each having different user data . one would work and executes the command and one ends failing with DataSource not found. and sometimes they both work.

i tried some of the suggestions in the troubleshooting section in this blog by using cloud-init as the customization engine instead . https://blah.cloud/infrastructure/using-cloud-init-for-vm-templating-on-vsphere/

Everything went okay except the DataSource ends being seed=VMwareTools and not OVF since vRA map the user-data as an iso connected to the VM CD-ROM device, but again cloud-init ends not executing anything but customization works.

[sqqqrly]

I am able to deploy ubuntu 18.04 to vSphere using terraform v0.13.5 and using cloud-init to configure the network.

Adding "After=dbus.service" to /lib/systemd/system/open-vm-tools.service fixed it.

I have an ansible role for installing open-vm-tools (from packer) and added this after installing the package:

      18 # Fix network interface not having a carrier.
      19 - name: Fix missing network error
      20   lineinfile:
      21     path: /lib/systemd/system/open-vm-tools.service
      22     insertafter: "After=apparmor.service"
      23     line: "After=dbus.service"

[PengpengSun]

Hi @sqqqrly

I am able to deploy ubuntu 18.04 to vSphere using terraform v0.13.5 and using cloud-init to configure the network.

Were you using cloud-init as customization engine which is described in this KB https://kb.vmware.com/s/article/54986 ?

I'm asking this since adding 'After=dbus.service' to open-vm-tools.service and customizing VM by cloud-init creates a dependency cycle for systemd:

• dbus has DefaultDependencies which makes it after sysinit.target
• Cloud-init-local needs to be early to make changes required before certain targets start e.g. Before=sysinit.target
• But open-vm-tools is before cloud-init-local

[sqqqrly]

@PengpengSun My Terraform deploy is similar to this example: https://github.com/grantorchard/terraform-vsphere-cloudinit-simple

I install open-vm-tools (modified service file) and cloud-init using packer to build an image. Later, Terraform then deploys the image and cloud-init runs using the extra_config data found in main.tf.

[PengpengSun]

@sqqqrly Great, so you are using https://github.com/vmware/cloud-init-vmware-guestinfo to deploy data to cloud-init, with that please ignore my previous comment on dependency cycle, they are different process.