anatol / booster

Fast and secure initramfs generator
MIT License
480 stars 39 forks source link

Booster does not work with drives formatted with 4K sectors #119

Closed anatol closed 2 years ago

anatol commented 2 years ago

Forking discussion off this reddit thread https://www.reddit.com/r/archlinux/comments/reqxkn/comment/hohidou/?utm_source=share&utm_medium=web2x&context=3

A user has following GPT table

# gdisk -l /dev/nvme0n1
GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/nvme0n1: 250051158 sectors, 953.9 GiB
Model: m.2 Smartbuy PS5012-2280 1024GB         
Sector size (logical/physical): 4096/4096 bytes
Disk identifier (GUID): 4F509632-605D-4E61-AACA-E1B0951584D6
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 5
First usable sector is 6, last usable sector is 250051152
Partitions will be aligned on 256-sector boundaries
Total free space is 425 sectors (1.7 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1             256          131327   512.0 MiB   EF00  EFI system partition
   2          131328        16908543   64.0 GiB    8300  Linux filesystem
   3        16908544       174194943   600.0 GiB   8300  Linux filesystem
   4       174194944       241662544   257.4 GiB   8300  Linux filesystem
   5       241662720       250051152   32.0 GiB    8200  Linux swap
# blkid
/dev/nvme0n1p5: UUID="ca30eb5c-6d2b-435e-a04b-df49c8954ad8" TYPE="swap" PARTLABEL="Linux swap" PARTUUID="d053ca67-488c-4514-a89e-1f74bf56d7a1"
/dev/nvme0n1p3: UUID="062d43d4-eb60-431a-b7e7-50aef43126b7" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Linux filesystem" PARTUUID="e26e1e01-c708-475f-86dd-617de2d5ebe6"
/dev/nvme0n1p1: UUID="4A9C-B17A" BLOCK_SIZE="4096" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="5301b67d-e872-468e-b487-469ad9bcc3c1"
/dev/nvme0n1p4: UUID="5482feff-4c59-45bf-bcaf-900eb55738a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Linux filesystem" PARTUUID="63dd8787-33ac-470e-9ae1-887578cf18bb"
/dev/nvme0n1p2: UUID="5e55a574-24c0-4f40-8255-c70a0f3d5dcd" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Linux filesystem" PARTUUID="50173cf6-3018-4924-8b91-a5e5d6e273ae"

For some reason it does not detect it as GPT.

cc @xshhhh

anatol commented 2 years ago

Here is the GPT detection logic https://github.com/anatol/booster/blob/f5424153991473595b2947808b30be0b80b21e47/init/blkinfo.go#L66

hxss commented 2 years ago

I'll duplicate here. Partition table created as always:

# gdisk /dev/nvme0n1
- create new GPT table
- create 512M EF00 partition for fat32
- create 8300 root/home/etc partitions for ext4

> w

# mkfs.fat -F 32 /dev/nvme0n1p1
# mkfs.ext4 /dev/nvme0n1p2

This script works on all my UEFI machines.

anatol commented 2 years ago

So I have a unit test for GPT detection

https://github.com/anatol/booster/blob/f5424153991473595b2947808b30be0b80b21e47/init/blkinfo_test.go#L79

that creates a partition table as

gdisk assets/gpt <<< 'o
y
x
g
c26fcabe-8010-4bff-a066-8c73e76dbb32
m
n

+2M
cb3451b1-35b0-40f9-a0ea-8566ed05de6d
c
сектор1
x
c
53453a07-9bee-4397-a614-9e8fa3870a72
m
n

+2M
cb3451b1-35b0-40f9-a0ea-8566ed05de6d
c
2
hello
x
c
2
3ea18db9-691f-455e-a40d-20d426b1965f
a
2
60

w
y
'
$ gdisk -l gpt
GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk gpt: 20480 sectors, 10.0 MiB
Sector size (logical): 512 bytes
Disk identifier (GUID): C26FCABE-8010-4BFF-A066-8C73E76DBB32
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 20446
Partitions will be aligned on 2048-sector boundaries
Total free space is 12221 sectors (6.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            6143   2.0 MiB     FFFF  сектор1
   2            6144           10239   2.0 MiB     FFFF  hello

The test/code works fine and detects the disk correctly as GPT.

The main difference I see is that your sector size is Sector size (logical/physical): 4096/4096 bytes while booster expects GPT sectors are 512 bytes. I'll need to test the code for sectors other than 512 bytes.

anatol commented 2 years ago

GPT header information is located at the first sector. But the sector might be different at different systems. How does software expect to figure out the sector size? I am looking at https://wiki.osdev.org/GPT "Protective Master Boot Record" section but it does not tell what is the size of the sector.

anatol commented 2 years ago

gdisk tool uses following ioctl to figure out the sector size:

and then operates with GPT sectors at logical size. It is what booster should do as well.

hxss commented 2 years ago

ah yes, I changed sector size to 4096b by nvme format

anatol commented 2 years ago

I was able to make a test that shows the problem with 4K logical sectors. And made a fix for this problem, it has been pushed to wip branch.

Please try branch wip and let me know if it fixes your problem @hxss

hxss commented 2 years ago

It works! And booster realy loads system 20% faster(from button press to cursor draw). Thank you for fast fix!

Unfortunately, booster does not really support the amdgpu module.

anatol commented 2 years ago

Thanks for the confirmation. The numbers actually say that booster is 8 times faster (initrd stage 1350 ms vs 155 ms).

For the amd gpu please file a separate github issue and let's debug it there.

hxss commented 2 years ago

Yeah, its 8 times and loader 3 times faster on paper. But I also checks real load time from power button press to cursor display with stopwatch. I tested many mkinitcpio configurations and tried to move/disable systemd services - its all gives different results in reports, but real time for mkinitcpio is always 9.6-9.8s. Where 4.1s is constant for UEFI load. For now booster shows 8.3s in real load time tests. Which is realy great result for me. And I'll also play with booster's modules configuration. Although most of the time is taken up by systemd.