hook-mdev: better `/dev/disk/by-id` via custom mdev script for wwid parsing and fallbacks

rpardini commented 5 months ago

hook-mdev: add `persistent-storage` script from upstream Alpine (ipsis-literis)

original persistent-storage from upstream Alpine:
- source https://gitlab.alpinelinux.org/alpine/mdev-conf/-/blob/master/persistent-storage?ref_type=heads
- revision a21d1053dd5ca37538b0435d66c58a30cb273658
this is being added as reference for future rebases; it will be shellfmt'ed and modified in later commits

hook-mdev: rename to `persistent-storage.sh` and shellfmt it (no real changes)

simple rename & shellfmt; separate commit for easy future rebasing

hook-mdev: add customized `persistent-storage` script which tries harder to produce /dev/disk/by-id entries by parsing `wwid`

this is still not-even-close to systemd's udev, but should at least add something to by-id when Alpine's mdev wouldn't
- if device reports model and serial, all was/is good;
- introduce:
- if model or serial missing, try parsing them from wwid;
  - falls back to using the sanitized wwid as serial if parsing fails
- last resort: falls back to using 'noserial' as serial for devices that only have a model.
also adds util-linux related apks, which brings a more capable blkid
for context: https://gitlab.alpinelinux.org/alpine/mdev-conf/-/commits/master/?ref_type=HEADS
add a log of logging, which you can see with cat /var/log/mdev.log on Hook's console

rpardini commented 5 months ago

Before this PR (Hook `hook-default-amd64` via LK's qemu)

/dev/disk
├── by-id
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

On Ubuntu, with systemd's udev:

/dev/disk
├── by-id
│   ├── ata-QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-0QEMU_QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-1ATA_QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   └── scsi-SATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
├── by-path
│   ├── pci-0000:00:1f.2-ata-1 -> ../../sda
│   ├── pci-0000:00:1f.2-ata-1-part1 -> ../../sda1
│   ├── pci-0000:00:1f.2-ata-1-part14 -> ../../sda14
│   ├── pci-0000:00:1f.2-ata-1-part15 -> ../../sda15
│   ├── pci-0000:00:1f.2-ata-1.0 -> ../../sda
│   ├── pci-0000:00:1f.2-ata-1.0-part1 -> ../../sda1
│   ├── pci-0000:00:1f.2-ata-1.0-part14 -> ../../sda14
│   ├── pci-0000:00:1f.2-ata-1.0-part15 -> ../../sda15
│   ├── pci-0000:00:1f.2-ata-3 -> ../../sr0
│   └── pci-0000:00:1f.2-ata-3.0 -> ../../sr0
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

With this PR (Hook `hook-default-amd64` via LK's qemu)

/dev/disk
├── by-id
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── ata-ro-QEMU_HARDDISK_QM00001 -> ../../sr0
│   ├── scsi-ro-t10.ATA_____QEMU_DVD-ROM____________________________QM00005 -> ../../sr0
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001 -> ../../sda
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part1 -> ../../sda1
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part14 -> ../../sda14
│   └── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

rpardini commented 5 months ago

Still not good enough for my USB-connected disks. Those have wwid with some naa., have a model sysfs file (which is being ignored) but don't have a serial sysfs. Drafted PR until minimally addressed

rpardini commented 5 months ago

Force-pushed fixed version. I had made a gross mistake (hardcoded sda) on the original commits; added a few fallbacks for disks without model / serial / wwid in that order, eventually using 'noserial' hardcoded.

This makes most disk devices at least appear in disk/by-id, even if they most probably won't match systemd's udev nomenclature, at least we have 'some' id. Using /dev/sdb et al directly is a recipe for disaster.

rpardini commented 5 months ago

This PR (after fixes), qemu

/dev/disk
├── by-id
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── ata-ro-QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-ro-t10.ATA_____QEMU_DVD-ROM____________________________QM00005 -> ../../sr0
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001 -> ../../sda
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part1 -> ../../sda1
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part14 -> ../../sda14
│   └── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

A different VM, using virtio disks with explicitly-set serial numbers:

/dev/disk
├── by-id
│   ├── virtio-cidata_serial -> ../../vda
│   └── virtio-root_disk_serial -> ../../vdb
├── by-label
│   └── cidata -> ../../vda
└── by-uuid
    └── 2024-06-01-11-43-11-85 -> ../../vda

tinkerbell / hook