error rendering template: map has no entry for key \"etcd_scheme\"

[KingJ]

What keywords did you search in tectonic-installer issues before filing this one?

• error rendering template
• etcd_scheme

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

• Tectonic version (release or commit hash): 1.7.1-tectonic.1
• Terraform version (terraform version): Unknown
• Platform (aws|azure|openstack|metal): metal

What happened?

When booting the first controller node, it correctly boots over PXE and starts CoreOS. CoreOS is then installed to disk and the machine reboots. However, when the machine reboots initrd fails as an invalid Ingition config is received. The Ignition configuration received from Matchbox is "404 page not found" - which matches the issue at https://github.com/coreos/matchbox/issues/463 whereby an a 404 is returned when a template parsing error occours.

As the machine's configuration via Ignition fails, it becomes stuck in a reboot loop and the Tectonic installer hangs on null resource etcd secrets still creating indefinitely.

The full error output in Matchbox's log is;

Aug 14 21:45:13 matchbox rkt[740]: [ 6640.170833] matchbox[5]: time="2017-08-14T21:45:13Z" level=error msg="error rendering template: template: :12:54: executing \"\" at <.etcd_scheme>: map has no entry for key \"etcd_scheme\""

Looking at the files in /var/lib/matchbox/groups, I see no references to etcd configuration. For example, here is fc-test-master.json;

{
    "id": "fc-test-master",
    "profile": "tectonic-worker",
    "selector": {
        "mac": "00:0c:29:f1:b6:2d",
        "os": "installed"
    },
    "metadata": {
        "cni_bin_dir_flag": "",
        "domain_name": "test-master.fc.kj",
        "k8s_dns_service_ip": "10.3.0.10",
        "kube_version_image": "quay.io/coreos/kube-version:0.1.0",
        "kubelet_image_tag": "v1.7.1_coreos.0",
        "kubelet_image_url": "quay.io/coreos/hyperkube",
        "ssh_authorized_key": "ssh-rsa REMOVED"
    }
}

What you expected to happen?

I expected the machine to receive an Ignition configuration and configure itself as a controller node.

How to reproduce it (as minimally and precisely as possible)?

Run through a Tectonic GUI install, opting to install etcd on the controller or on the cluster (both result in the error).

Anything else we need to know?

Although this is a bare metal install, the machines themselves are VMs. Likely irrelevant though!

Many thanks.

[KingJ]

I think this might have been caused by left-over files in /var/lib/matchbox/groups from a previous attempt at installing Tectonic a few months ago with machines that had the same MAC address - hence it looks like Matchbox was attempting to template older files. After removing the entire contents of /var/lib/matchbox/groups and re-starting the installer Matchbox is now serving up Ignition configs.

[robszumski]

Awesome, glad you figured it out. Closing, but reopen if needed.