Open ChrisPbb opened 2 days ago
good catch!
from diskfs
// when we use a disk image with a GPT, we cannot get the logical sector size from the disk via the kernel
//
// so we use the default sector size of 512, per Rod Smith
ahhhh this is a RAID, interesting!
Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
I wonder why a RAID has that by default :D
patch sent and tested.
To test, run qemu with a custom nvme disk with 4096 sector size:
$ qemu-img create /tmp/disk.img +50G
$ qemu-system-x86_64 -cpu host -smp 4 -m 8192M -name test -serial file:/tmp/serial.out -device e1000,netdev=user.0 -machine type=pc,accel=kvm -netdev user,id=user.0,hostfwd=tcp::2222-:22 -cdrom kairos-ubuntu-24.04-standard-amd64-generic-v3.2.1-23-g409dc0d.iso -drive file=/tmp/disk.img,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1,physical_block_size=4096,logical_block_size=4096
The try with the patched version and see that it generates the proper sizes:
Disk /dev/nvme0n1: 50 GiB, 53687091200 bytes, 13107200 sectors
Disk model: QEMU NVMe Ctrl
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 2CEFF841-4C1D-5A14-8E46-ED3AEB621093
Device Start End Sectors Size Type
/dev/nvme0n1p1 256 511 256 1M BIOS boot
/dev/nvme0n1p2 512 16895 16384 64M Linux filesystem
/dev/nvme0n1p3 16896 1011711 994816 3.8G Linux filesystem
/dev/nvme0n1p4 1011712 2683135 1671424 6.4G Linux filesystem
/dev/nvme0n1p5 2683136 13106943 10423808 39.8G Linux filesystem
Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
Some NVMe drives are byte-addressable (sector size = 1). Ok technically, an NVMe drive can have a different sector size for each namespace. But in any case, assuming these things is bad and not sustainable over time. For example, SSD vendors are pushing for minimum block size to be 16KiB for their more dense products. So it is best to move away from counting sectors and towards either a reasonably appropriate minimum size or percentage. In this case, the RAID vendor is lying about the minimum I/O size and probably other things, too... quite common, unfortunately. Lots of mines in this minefield.
As for this exact problem, elemental.PartitionAndFormatDevice
shouldn't assume partition.Read
only returns a gptTable
because it could return an mbrTable
You can probably work around it by putting a GPT label on the disk.
Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
Some NVMe drives are byte-addressable (sector size = 1). Ok technically, an NVMe drive can have a different sector size for each namespace. But in any case, assuming these things is bad and not sustainable over time. For example, SSD vendors are pushing for minimum block size to be 16KiB for their more dense products. So it is best to move away from counting sectors and towards either a reasonably appropriate minimum size or percentage. In this case, the RAID vendor is lying about the minimum I/O size and probably other things, too... quite common, unfortunately. Lots of mines in this minefield.
Interesting!
As for this exact problem,
elemental.PartitionAndFormatDevice
shouldn't assumepartition.Read
only returns agptTable
because it could return anmbrTable
You can probably work around it by putting a GPT label on the disk.
mmmh, we only support GPT partition tables in Kairos, if the disk has a MBR then it should fail with a clear error message saying that we dont support it.
I think the error that diskfs returned was mainly because we screwed with the sector size and it returned a wrong or nil object. We hardcoded the sector size, instead of opening the disk and letting it tell us the actual sector size. So there has to be a missing error check either on our side or on diskfs that makes the return value not raise an error.
In any case, the patch attached should alleviate this issue :D
Kairos version:
CPU architecture, OS, and Version:
Describe the bug The install process creates partition tables as if the sector size was 512B even when the sector size is 4096B. This does not happen with v3.1.1
To Reproduce Install kairos on a disk device with sector size 4096B. Any configuration (except
install.no-format: true
) will produce the same results.Expected behavior Partitions are created with sector size matching the disk device's physical block size.
Logs Here's the actual error during install (OCRd from console screenshot)
Here's evidence of the misaligned partitions (OCRd from console screenshot)