user-guides: add platform docs for vmware

[lucab]

We currently don't have any documentation on how to prepare, customize and first-boot FCOS on the vmware platform.

We can start by porting over the docs from Container Linux at https://coreos.com/os/docs/latest/booting-on-vmware.html#vmware-guestinfo-interface.

More interestingly, now there seems to be an official PowerShell based container which should even support setting guestinfo.

[lucab]

Self note: Import-VApp does not seem to support remote HTTP source. An upload from a local source path seems required.

[lucab]

Self note: I got a simple flow working with OVA importing (after https://github.com/coreos/coreos-assembler/pull/1141), steps here below.

# OVA is locally at ./templates/fcos.ova
podman run --rm -it -v ./templates:/root/templates:z vmware/powerclicore:latest

# Interactive authentication
Connect-VIServer -Server vsphere.example.com

# Host machine
$vmwHost = 'host1.example.com'
# Path to OVA (in container)
$ovaPath = '/root/templates/fcos.ova'
# VM name
$nodeName = 'fcos'
# Ignition config (after `base64 -w0`)
$nodeConfig = @{}
$nodeConfig["guestinfo.ignition.config.data.encoding"] = "base64"
$nodeConfig["guestinfo.ignition.config.data"] = "<Ignition config in base64>"

Import-VApp -VMHost $vmwHost -Name $nodeName -OvfConfiguration $nodeConfig -Source $ovaPath

Start-VM $nodeName

[lucab]

Before committing to this PowerCLI approach, I'd like to explore https://github.com/vmware/govmomi/blob/master/govc/README.md too. On paper, it may integrate better with the rest of our Golang world.

[lucab]

Equivalent steps using govc and storing the OVA as a library template:

mkdir templates
coreos-installer download -s stable -p vmware -f ova -C ./templates/

govc session.login -u 'user:pass@vsphere.example.com'
govc library.create fedora-coreos
govc library.import -n fcos-stable fedora-coreos ./templates/fedora-coreos-<VER>-vmware.x86_64.ova

govc library.deploy fedora-coreos/fcos-stable fcos-stable-worker01
govc vm.change -vm fcos-stable-worker01 -e "guestinfo.ignition.config.data.encoding=base64"
govc vm.change -vm fcos-stable-worker01 -e "guestinfo.ignition.config.data=<Ignition config in base64>"
govc vm.info -e fcos-stable-worker01

govc vm.power -on fcos-stable-worker01

[bgilbert]

We may want to document that deploying the OVA from vSphere directly to an ESX host will not work without some extra contortions.

[lucab]

@bgilbert the easy bits covering vShere are up at https://github.com/coreos/fedora-coreos-docs/pull/57. I honestly got lost trying to follow the discourse thread, so I don't have a good view on how the "deploy directly to ESX host" flow would look like. For now, I just kept the structure of that page in way that it should be easier to add more "Booting a new VM on XYZ" later on.

[Okeanos]

@bgilbert , @lucab is there any particular information still missing here from your point of view? I have access to VMWare Fusion & a vSphere instance that could be used to test some things.

From my side, having deployed FCOS instances with a broken Ignition config to vSphere, I can safely say that some additional debugging information would be handy. For example, one can extract the boot/system log from an FCOS VM by adding a serial port device to the VM. vSphere will then create a user downloadable file containing all information FCOS prints during booting/start up.

Also, govc has problem with "large" Ignition configs depending on the platform (see vmware/govmomi/issues/2488 for more details) it is being used on. Something I think warrants inclusion in the FCOS documentation as well.

Beyond that using additional disks, in particular reusing them between FCOS instances, is as far as I can currently tell, a little weird. Creating and attaching a disk to the VM is fairly straightforward. However, detaching is only possible via govc (no GUI option for that available) and must be done before deleting a VM if one wants to keep it. vSphere will delete all files, including separately created disks, if the VM they are attached to is deleted.