udev: Provide stable correlateable device symlinks for specific VM environments

[tasleson]

Is your feature request related to a problem? Please describe. Users provisioning VM instances would like the ability to correlate a block device in the VM management interface outside of the VM with that within the VM instance. This is useful for writing kick start files to auto provision new instances.

Describe the solution you'd like

A udev rules file and associated helper that creates symlinks that offers a stable way to correlate.

Describe alternatives you've considered

One alternative is to use disk size to distinguish disks in a kick start.

%pre --interpreter /bin/bash --log /tmp/ks_pre.log
correlateable
    # Dump whole SCSI/IDE disks out sorted from smallest to largest ouputting
    # just the name
    disks=(`lsblk -n -o NAME -l -b -x SIZE -d -I 8,3`) || exit 1

    # We are assuming we have 3 disks which will be used
    # and we will create some variables to represent
    d0=${disks[0]}
    d1=${disks[1]}
    d2=${disks[2]}

    echo "part /home --fstype="xfs" --ondisk=$d2 --grow" >> /tmp/disks
    echo "part swap --fstype="swap" --ondisk=$d0 --size=4096" >> /tmp/disks
    echo "part / --fstype="xfs" --ondisk=$d1 --grow" >> /tmp/disks
    echo "part /boot --fstype="xfs" --ondisk=$d1 --size=1024" >> /tmp/disks

%end

This doesn't work when the disks have identical sizes.

The systemd version you checked that didn't have the feature you are asking for

Feature does not exist in any version.

I've written a POC which partially addresses this for hyper-V, https://github.com/tasleson/misc-tools/tree/master/vmblkid . It would need to be expanded to handle "Synthetic IDE controller" too. Testing shows it works when the host portion of the HBTL changes. I would also be good to expand functionality to other hypervisors too.

My main questions are ...

1. Does something like this seem reasonable?
2. If the answer to 1 is yes, what would be the most acceptable implementation for inclusion if a PR was done?

[poettering]

how are the VM managers supposed to communicate their ids?

why don't the current path IDs suffice for this?

[poettering]

@bluca you work with azure people, is there any infra for this?

[bluca]

Don't know, will ask around

[bluca]

@poettering actually I'm not sure I understand the question - what are we looking for exactly?

[poettering]

@tasleson forwarding @bluca's question back to you?

[tasleson]

@bluca I believe @poettering was asking if you have access to hyper-V VMs to do testing/development on.

As for more background on this issue. A user creates a VM using the hyper-V GUI. In that GUI they can create 1 or more adapters and each adapter can have 0-64 virtual disk drives. Depending on the probe sequence of the virtual adapters in the guest VM, the Host portion of the HBTL (aka. HCTL) changes. From the GUI you can see a numbering of adapter N, disk N etc. What users would like is the ability to know that adapter 0, disk 2 will be my root FS and that adapter 1 disk 0 will be my home directory etc. from looking at the GUI and then specifying which disk this is the VM. The problem is there isn't a consistent way to correlate the two because of the host portion of the HBTL changes from boot to boot, everything else remains consistent.

I outlined one workaround where you select the disk by size in the kickstart, doesn't work for every case.

The proof of concept I've referenced in the issue, leverages the sysfs information that is exposed to the guest VM to ensure that the HBTL consistently matches the end users view of the world outside of the VM and allows for another type of symlink to be created.

I'm not asking for systemd developers to implement this. I'm asking if someone else was to implement it, what would be the implementation be that would be most acceptable to you all. Do you want a compiled helper application and associated udev rules file, add functionality to an existing udev helper and rules file etc.?

[arvidjaar]

Depending on the probe sequence of the virtual adapters in the guest VM, the Host portion of the HBTL (aka. HCTL) changes.

If you talk about default SCSI ID handling, path_id tries to compute host number relative to parent device. See https://github.com/systemd/systemd/blob/4917c15af7c2dfe553b8e0dbf22b4fb7cec958de/src/udev/udev-builtin-path_id.c#L317-L331

Also path_id already supports Hyper-V vmbus: https://github.com/systemd/systemd/blob/4917c15af7c2dfe553b8e0dbf22b4fb7cec958de/src/udev/udev-builtin-path_id.c#L372

If this is not enough, you should work off existing code and improve it. My understanding is the resulting path is unique and stable (for a given VM), but I am not sure whether GUID is easily available in Hyper-V management interface.

[tasleson]

My understanding is the resulting path is unique and stable (for a given VM), but I am not sure whether GUID is easily available in Hyper-V management interface.

This would require the automated install to have a custom crafted kickstart file for each newly created VM. Yes, this is indeed possible to automate, but it increases the complexity considerably.

This powershell script which you run on the hyper-v host provides what the wwn would be for a specific SCSI disk, but suffers the same problem.

# Output what the /dev/disk/by-id/ for the specified hyper-v virtual disk.
# Takes a single parameter which is the virtual disk file.
param (
    [Parameter(Mandatory=$true)][string]$virtualdisk
)

$what = Get-VHD -Path $virtualdisk
$part = $what.DiskIdentifier.ToLower().split('-')

$p = $part[0]
$s0 = $p[6] + $p[7] + $p[4] + $p[5] + $p[2] + $p[3] + $p[0] + $p[1]

$p = $part[1]
$s1 =  $p[2] + $p[3] + $p[0] + $p[1]

[string]::format("/dev/disk/by-id/wwn-0x60022480{0}{1}{2}", $s0, $s1, $part[4])