Installer assumes disk is called /dev/sda

[abh]

Tectonic Version

1.4.7

Environment

libvirt

Expected Behavior

Installation of CoreOS to disk

Actual Behavior

After PXE booting the virtual machine successfully the progress stopped because the installer couldn't find /dev/sda.

Reproduction Steps

PXE boot libvirt machines using bootcfg.

Other Information

After replacing /dev/sda with /dev/vda in the bootcfg files in /var/lib/bootcfg, restarting bootcfg (not sure if that was necessary) and PXE booting again the installation proceeded as expected.

[mfburnett]

Hey @abh, thanks for filing this issue! So today, we hardcode using the /dev/sda hard-drive when installing Tectonic on bare metal. Enabling configurability around this is something we're going to look into as we build out functionality for installing on virtual machines. I'll make a note to return to this issue when we further scope out that functionality!

[dghubble]

@abh For now, substituting /dev/sda in the /var/lib/bootcfg/ignition/install-reboot.json file is the right way to go. This does not require bootcfg to be restarted.

[tnsasse]

I am having the same issue on xenserver, the installer is trying to install to /dev/sda where it should be /dev/xvda and silently fails. Journalctl on the machine gave me the hint. Trying the workaround now...

[mfburnett]

@tnsasse did the workaround work for you?

[dghubble]

In the new terraform-based installer, it is best to just set the disk as a variable. You do not need to modify matchbox state. https://github.com/coreos/tectonic-installer/blob/master/platforms/metal/cl/coreos-install.yaml.tmpl#L24

Tectonic should implement this as a customization variable.

[tnsasse]

@mfburnett changing the disk names in the templates worked for me.

[alexandrufulop]

@dghubble Thank you!

[dghubble]

I'll point out the Matchbox Kubernetes example supports this customization. https://github.com/coreos/matchbox/tree/master/examples/terraform/bootkube-install#optional It may be valuable for Tectonic to support the same.

[TerraTech]

If you are running heterogeneous systems, e.g. a combo of metal and VM systems, then the _installdisk variable needs to be on a per-node basis.

We've hacked up our installer to handle per-node configuration options as some need special serial console handling, amongst other things.

Here is part of the special core-metal.tfvars that we use:

core-metal_map = {
    "core-minion01" = {
        name = "core-minion01"
        int_vlan111_mac = "xx:xx...."
        int_vlan111_address = "x.x.x.x/x"
        is_minion = "true"
        is_virt = "true"
        install_disk = "/dev/vda"
        power_handling = "snapshot"
        sc_primary = "false"
        want_swap = "1G"
    }
...
}

Some keys like _isminion (or _ismaster) are used in {{ if ... }}...{{end}} template blocks as we have tried to DRY our templates. Other keys like _powerhandling is used during development so we can take snapshots between installation steps so we can just focus on the stage we are tweaking on subsequent runs.

We will use the above map as a lookup when setting the metadata:

resource "matchbox_group" "install-core-metal" {
    count = "${length(var.core-metal_map)}"
    ...
    metadata {
        ...
        install_disk = "${lookup(var.core-metal_map[element(keys(var.core-metal_map), count.index)], "install_disk", var.install_disk)}"
        ...
    }
}

Unfortunately, per-node becomes invasive pretty quick due to the nature of how terraform/HCL works as the existing _resource "matchbox_group" "coreosinstall" is pretty much a one-size fits all implementation.

If the tectonic-installer developers are open to a PR on this, I can put in the effort with reworking "coreos_install" to be per-node capable, with built-in defaults for unset keys.