Itxaka
commented
1 day ago good catch!
Open ChrisPbb opened 2 days ago
Itxaka
commented
1 day ago good catch!
Itxaka
commented
1 day ago from diskfs
// when we use a disk image with a GPT, we cannot get the logical sector size from the disk via the kernel
//
// so we use the default sector size of 512, per Rod Smithahhhh this is a RAID, interesting!
Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
I wonder why a RAID has that by default :D
Itxaka
commented
1 day ago patch sent and tested.
To test, run qemu with a custom nvme disk with 4096 sector size:
$ qemu-img create /tmp/disk.img +50G
$ qemu-system-x86_64 -cpu host -smp 4 -m 8192M -name test -serial file:/tmp/serial.out -device e1000,netdev=user.0 -machine type=pc,accel=kvm -netdev user,id=user.0,hostfwd=tcp::2222-:22 -cdrom kairos-ubuntu-24.04-standard-amd64-generic-v3.2.1-23-g409dc0d.iso -drive file=/tmp/disk.img,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1,physical_block_size=4096,logical_block_size=4096The try with the patched version and see that it generates the proper sizes:
Disk /dev/nvme0n1: 50 GiB, 53687091200 bytes, 13107200 sectors
Disk model: QEMU NVMe Ctrl
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 2CEFF841-4C1D-5A14-8E46-ED3AEB621093
Device Start End Sectors Size Type
/dev/nvme0n1p1 256 511 256 1M BIOS boot
/dev/nvme0n1p2 512 16895 16384 64M Linux filesystem
/dev/nvme0n1p3 16896 1011711 994816 3.8G Linux filesystem
/dev/nvme0n1p4 1011712 2683135 1671424 6.4G Linux filesystem
/dev/nvme0n1p5 2683136 13106943 10423808 39.8G Linux filesystem
richardelling
commented
1 day ago Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
Some NVMe drives are byte-addressable (sector size = 1). Ok technically, an NVMe drive can have a different sector size for each namespace. But in any case, assuming these things is bad and not sustainable over time. For example, SSD vendors are pushing for minimum block size to be 16KiB for their more dense products. So it is best to move away from counting sectors and towards either a reasonably appropriate minimum size or percentage. In this case, the RAID vendor is lying about the minimum I/O size and probably other things, too... quite common, unfortunately. Lots of mines in this minefield.
As for this exact problem, elemental.PartitionAndFormatDevice shouldn't assume partition.Read only returns a gptTable because it could return an mbrTable You can probably work around it by putting a GPT label on the disk.
Itxaka
commented
19 hours ago Last thing I know was that most of (some?) nvme drives have a 512 sector size compatibility layer that they show to the OS, even if underlying the size is 4096.
Some NVMe drives are byte-addressable (sector size = 1). Ok technically, an NVMe drive can have a different sector size for each namespace. But in any case, assuming these things is bad and not sustainable over time. For example, SSD vendors are pushing for minimum block size to be 16KiB for their more dense products. So it is best to move away from counting sectors and towards either a reasonably appropriate minimum size or percentage. In this case, the RAID vendor is lying about the minimum I/O size and probably other things, too... quite common, unfortunately. Lots of mines in this minefield.
Interesting!
As for this exact problem,
elemental.PartitionAndFormatDeviceshouldn't assumepartition.Readonly returns agptTablebecause it could return anmbrTableYou can probably work around it by putting a GPT label on the disk.
mmmh, we only support GPT partition tables in Kairos, if the disk has a MBR then it should fail with a clear error message saying that we dont support it.
I think the error that diskfs returned was mainly because we screwed with the sector size and it returned a wrong or nil object. We hardcoded the sector size, instead of opening the disk and letting it tell us the actual sector size. So there has to be a missing error check either on our side or on diskfs that makes the return value not raise an error.
In any case, the patch attached should alleviate this issue :D
Kairos version:
CPU architecture, OS, and Version:
Describe the bug The install process creates partition tables as if the sector size was 512B even when the sector size is 4096B. This does not happen with v3.1.1
To Reproduce Install kairos on a disk device with sector size 4096B. Any configuration (except
install.no-format: true) will produce the same results.Expected behavior Partitions are created with sector size matching the disk device's physical block size.
Logs Here's the actual error during install (OCRd from console screenshot)
Here's evidence of the misaligned partitions (OCRd from console screenshot)