Raspberry pi4B: can't boot on usb+gpt with 4TB drive (all ok on 8G usbkey)

[LucaFulchir]

Description

On an rpi4 I broke 2 microsd due to (probably) too many writes.
I decided to go all usb and to try to make it a nas+other things, on it, so I have 2x4TB drives on usb3.

rpi4 has added support for usb boot on September, I have updated the eeprom and I can boot from a usb stick easily by flashing the latest successful unstable nixos build on hydra. No problems there apparently.

I can create a USB key (8Gb) with GPT and copy the installation media partitions there. It boots correctly.
I can not do the exact same steps on a 4TB disk, I get a lot of "gpt partition table header signature is wrong" and after a while of waiting for network boot I get sent to a u-boot prompt

I don't want to use MBR on the 4TB drives, since I would lose half the total space.

To Reproduce

• get a 4TB drive (probably anything over 2TB?)
• create empty GPT (I used gdisk from the latest nix hydra image)
• create 2 partitions, first 3G type ef00, second full free space type 8300
• mkfs.vfat -F 32 -n NIXOS_BOOT on the first partition, copy there the contents of the first partition of the latest working hydra image ( just a dump is probably the same)
• dump the second partition of the hydra image on the second partition on the drive

Expected behavior

Boots normally with a 8GB USB key, does not load the kernel on a 4TB drive

Additional Info

The errors "gpt partition table header signature is wrong" only appear on the nixos boot on the 4TB drive, so it might be a problem of the u-boot version.
Maybe the hydra image is using u-boot 32 bits and it can't handle disks bigger than 2TB? I don't know how to verify that. I don't know how to debug/patch/update u-boot, suggestions?

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rpi4-usb-boot-gpt/11440/3

[LucaFulchir]

Just for reference I tried:

• checking out master nixpkgs
• change uboot to 21.04
• recompile uboot with LBA48 and SYS_64_BIT_LBA flags:

file: pkgs/misc/uboot/default.nix

   ubootRaspberryPi4_64bit = buildUBoot {
     defconfig = "rpi_4_defconfig";
     extraMeta.platforms = ["aarch64-linux"];
+    extraMakeFlags = [ "CONFIG_LBA48=1" "CONFIG_SYS_64BIT_LBA=1" ];
     filesToInstall = ["u-boot.bin"];
   };

With a couple more things to let master compile the whole image for raspberry.
The theory was that those two flags should enable usage of disks > 2.1TB in uboot.
I am still unable to make it boot, the message is always the same, complaining about invalid GPT header signature

Suggestions on what I could try?

[jjthiessen]

I already commented on a related issue (https://github.com/NixOS/nixpkgs/issues/127814#issuecomment-873523810), as the errors that I am getting from U-Boot are the same as in the other issue (including the reported signature that, AFAICT, mysteriously doesn't appear in the partition table); however, my setup is more similar to yours (I'm trying to boot from a single 5TB USB drive).

[LucaFulchir]

I kind of gave up and I am using the SD for jsut the partitions for firmware + /boot everything else is on my 4TB drives, seems to work.

Not the setup I wanted, and I have not tried again since the last report in May,

I need someone with u-boot experience, the current compiled one clearly does not support drives > 2TB

[jjthiessen]

It's funny you say that, because I did the same thing (on July 4th or 5th :)).

[stale[bot]]

I marked this as stale due to inactivity. → More info