disk_setup could deal with non-deterministic device naming on EC2 nitro instances

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1828611

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = None
date_created = 2019-05-10T16:50:27.772455+00:00
date_fix_committed = None
date_fix_released = None
id = 1828611
importance = low
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1828611
milestone = None
owner = nils-nm
owner_name = Nils Meyer
private = False
status = triaged
submitter = nils-nm
submitter_name = Nils Meyer
tags = []
duplicates = []

Launchpad user Nils Meyer(nils-nm) wrote on 2019-05-10T16:50:27.772455+00:00

Cloud Provider: Amazon Web Services

As is documented in [1], instances on the nitro type hypervisor don't attach the NVME disks in deterministic order, yielding a different order of disks, example [2]. This makes it somewhat difficult to format and partition volumes since you don't know the volume ids beforehand when creating an instance (in an Autoscaling group for example).

My current thinking is that maybe a sort of special device name (much like swap / ephemeralX) coul be used to locate a device, for example ebs:root for the root drive - which is easy to detect, ebs:size=12G[0] for the first volume found with 12GiB size. With an appropriate instance profile and boto3 more elaborate selectors are conceivable (for example based on tags).

Further complicating things is that the metadata endpoint doesn't expose the correct device names, opting instead for fantasy names (sda1 for the root volume, sdX for other volumes).

My Workaround for a 2 volume instance: Try and format both devices, then mount the disk by label:

cloud-config

fs_setup:

• label: mylabel device: /dev/nvme0n1 filesystem: xfs partition: none overwrite: false
• label: mylabel device: /dev/nvme1n1 filesystem: xfs partition: none overwrite: false

mounts:

• ["/dev/disk/by-label/mylabel", "/mnt/label", "xfs", "defaults"]

[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html [2] https://gist.github.com/nilsmeyer/eddcfa4b7fc5b04ebc0be9eaa3c7b7dd

[ubuntu-server-builder]

Launchpad user Ryan Harper(raharper) wrote on 2019-07-18T21:09:06.397495+00:00

Thanks for filing the issue and providing your workaround.

[ubuntu-server-builder]

Launchpad user Jan Brouwers(janbrouwers) wrote on 2021-07-23T15:02:46.672198+00:00

running into the same issue. The workaround does not work for me as I need 2 drives formatted with different filesystems.

Worse, there doesn't seem to be a way to identify the drives from the ec2 metadata.