search

coreos / tectonic-installer

Install a Kubernetes cluster the CoreOS Tectonic Way: HA, self-hosted, RBAC, etcd Operator, and more
Apache License 2.0
601 stars 266 forks source link

vmware: CL support for OVF Environments #1729

Closed alekssaul closed 7 years ago

alekssaul commented 7 years ago

Versions

What happened?

Per Scott S. VMware Install failed against CL 1465.6.0 image as the CL image did not abide by not using DHCP provision as expected . Opening an issue for tracking.

edit: based on further digging it seems like new version of ignition (> 0.16) requires ignition configuration to be passed via OVF Environment interface instead of guestinfo

What you expected to happen?

Terraform inject static IP via ignition/systemd-networkd. CL Image should have been configured with static IP instead of DHCP Tectonic on VMware with CL 1465.6.0 image to be provisioned

How to reproduce it (as minimally and precisely as possible)?

Use Tectonic on VMware guide with CL 1465.6.0 OVA

Anything else we need to know?

Issue created for tracking purposes. Will be investigating further.

snsumner commented 7 years ago

I've confirmed within my own VMware lab environment that this version is ignoring the static IPs provided and using DHCP.

snsumner commented 7 years ago

Error applying plan:

4 error(s) occurred:

Terraform does not automatically rollback in the face of errors. Instead, your Terraform state file has been partially updated with any resources that successfully completed. Please address the error above and apply again to incrementally change your infrastructure.

alekssaul commented 7 years ago

/cc: @crawford to keep me honest. + @robszumski due to Tectonic on VMware impact

Tectonic for VMware is using guestinfo.coreos.config.data to pass ignition data into CoreOS CL OVA template. With the change in; https://github.com/coreos/ignition/commit/cb09b1bd9488378986c383a7ca2243f1536c11fd ignition will detect that OVF environment is in use and ignore guestinfo.coreos.config.data under the assumption that user either interactively (or via OVF spec) deployed the virtual machine.

Tectonic / Terraform workflow is stuck in the middle where OVF template is used, but no data is passed via OVF interface. Resulting in ignition not getting configuration data.

I can see 3 options to resolve:

  1. have the end user work around manually. When end-user imports CL image into vSphere, end user will have to manually un-check "Enable vApp Options" button. I've tested this with 1465.6.0 release with success. image

  2. have ignition attempt to fall back (or default) to guestinfo.coreos.config.data if env.Properties["guestinfo.coreos.config.data"], is empty. (Somewhere here; https://github.com/coreos/ignition/blob/v0.16.0/internal/providers/vmware/vmware_amd64.go#L68-L69)

  3. Add support to upstream Terraform to enable passing OVF spec into VM creation action and modify Tectonic Terraform to use the OVF Spec instead of guestinfo.coreos.config.data

IMHO quickest resolution is (option 1) to add documentation to disable "Enable vApp Options" into Step 2/3 of https://github.com/coreos/tectonic-installer/blob/master/Documentation/install/vmware/vmware-terraform.md#prerequsities. Option 3 would be a good long term goal.

snsumner commented 7 years ago

I confirmed in my ESXi 6.5u1 lab environment that workaround #1 to uncheck vApps option did resolve the issue.

crawford commented 7 years ago

@alekssaul What are the contents of the "ovfenv" variable when you un-check that option?

amarruedo commented 6 years ago

@alekssaul regarding option 1 mentioned in:

  1. have the end user work around manually. When end-user imports CL image into vSphere, end user will have to manually un-check "Enable vApp Options" button. I've tested this with 1465.6.0 release with success.

I cannot un-check "Enable vApp Options" button since that option is missing on my machines (we don't have appropriate license to enable DSR on our cluster). Does this mean that I cannot deploy CoreOS virtual machines on vSphere using Terraform? (I've tried with different versions of terraform-vsphere-provider, using custom_configuration_parameters and the newer extra_configs to pass guestinfo.coreos.config.data, but it is always ignored).