RPI 5: Problem with boot from USB 3.0 SDD on pieeprom-2024-01-22.bin

[BohdanBuinich]

Describe the bug

After updating to the latest EEPROM, I have a problem with the initialization SDD during kernel boot

[ 1.729779] usb 4-1: Enable of device-initiated U1 failed.

full log:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x414fd0b1]
[    0.000000] Linux version 6.1.74-rt22-ovos-buildroot (bohdan@bohdan) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.02.2-934-g68dab30c34-dirty) 12.3.0, GNU ld (GNU Binutils) 2.39) #1 SMP PREEMPT_RT Sun Jan 28 23:48:06 EET 2024
[    0.000000] random: crng init done
[    0.000000] Machine model: Raspberry Pi 5 Model B Rev 1.0
[    0.000000] earlycon: pl11 at MMIO 0x000000107d001000 (options '115200n8')
[    0.000000] printk: bootconsole [pl11] enabled
[    0.000000] efi: EFI v2.70 by worproject
[    0.000000] efi: SMBIOS=0x38500000 SMBIOS 3.0=0x384e0000 MEMATTR=0x3a3b8018 RNG=0x3867ff98 MEMRESERVE=0x38743e18 
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000014000000, size 512 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
.....
[    1.217331] netconsole: network logging started
[    1.217346] of_cfs_init
[    1.217372] of_cfs_init: OK
[    1.227179] uart-pl011 107d001000.serial: no DMA platform data
[    1.247534] mmc0: SDHCI controller on 1000fff000.mmc [1000fff000.mmc] using ADMA 64-bit
[    1.400831] mmc1: SDHCI controller on 1001100000.mmc [1001100000.mmc] using ADMA 64-bit
[    1.400883] Waiting for root device PARTUUID=c0932a41-44cf-463b-8152-d43188553ed4...
[    1.433268] mmc1: new ultra high speed DDR50 SDIO card at address 0001
[    1.670811] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
[    1.702512] usb 4-1: New USB device found, idVendor=0bda, idProduct=9210, bcdDevice=20.01
[    1.702514] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    1.702515] usb 4-1: Product: Ugreen Storage Device
[    1.702516] usb 4-1: Manufacturer: Ugreen
[    1.702517] usb 4-1: SerialNumber: 012938051A46
[    1.729779] usb 4-1: Enable of device-initiated U1 failed.
[    1.730275] usb 4-1: Enable of device-initiated U2 failed.
[    1.759941] usb 4-1: Enable of device-initiated U1 failed.
[    1.760428] usb 4-1: Enable of device-initiated U2 failed.
[    1.761452] scsi host0: uas
[    2.237613] scsi 0:0:0:0: Direct-Access     MIBRAND  CAIMAN 128GB SSD 1.00 PQ: 0 ANSI: 6
[    2.258808] sd 0:0:0:0: [sda] 250069680 512-byte logical blocks: (128 GB/119 GiB)
[    2.260077] sd 0:0:0:0: [sda] Write Protect is off
[    2.260079] sd 0:0:0:0: [sda] Mode Sense: 37 00 00 08
[    2.262565] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.263632] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[    2.263633] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes
[    2.272654]  sda: sda1 sda2 sda3 sda4 sda5
[    2.272806] sd 0:0:0:0: [sda] Attached SCSI disk
[    2.279664] VFS: Mounted root (squashfs filesystem) readonly on device 8:2.
[    2.280832] devtmpfs: mounted
[    2.283624] Freeing unused kernel memory: 4928K
[    2.314314] Run /sbin/init as init process
[    2.314316]   with arguments:
[    2.314316]     /sbin/init
[    2.314317]   with environment:
[    2.314317]     HOME=/
[    2.314318]     TERM=linux
[    2.314318]     BOOT_IMAGE=(hd0,gpt2)/boot/kernel
[    2.314319]     cgroup_memory=1
[    2.494439] systemd[1]: systemd 252 running in system mode (+PAM +AUDIT -SELINUX -APPARMOR -IMA -SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP -SYSVINIT default-hierarchy=unified)
[    2.494446] systemd[1]: Detected architecture arm64.

after this, the boot is stuck.

Also after several reboots, I'm able to login to the system, but after 5 minutes I always see a random SSD disconnect [ 2025.215789] EXT4-fs (sda5): Remounting filesystem read-only

I tried to add USB SDD to the quirks list, but it did not help me NOTE: it works fine when I connected SDD to USB 2.0

When I downgraded to pieeprom-2024-01-05.bin - all works fine

Steps to reproduce the behaviour

1. Flash system to USB SDD with new pieeprom-2024-01-22.bin
2. Connect to USB 3.0 port on RPI 5

Device (s)

Raspberry Pi 5

Bootloader configuration.

[all] BOOT_UART=1 POWER_OFF_ON_HALT=0 BOOT_ORDER=0xf461

System

No response

Bootloader logs

T: Bus=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 2 Spd=5000 MxCh= 0 D: Ver= 3.20 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 9 #Cfgs= 1 P: Vendor=0bda ProdID=9210 Rev=20.01 S: Manufacturer=Ugreen S: Product=Ugreen Storage Device S: SerialNumber=012938051A46 C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=896mA I: If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage E: Ad=81(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms E: Ad=02(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms I: If#= 0 Alt= 1 #EPs= 4 Cls=08(stor.) Sub=06 Prot=62 Driver=usb-storage E: Ad=81(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms E: Ad=02(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms E: Ad=83(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms E: Ad=04(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms

USB boot

No response

NVMe boot

No response

Network (TFTP boot)

No response

[timg236]

Since the Linux kernel has it's own independent USB stack and does not inherit state / code from the EEPROM this doesn't sound right. I suggest trying to reproduce this on a 6.6 RPi OS kernel

[lurch]

I suggest trying to reproduce this on a 6.6 RPi OS kernel

...which you can get with sudo rpi-update next. See https://forums.raspberrypi.com/viewtopic.php?t=361116 for more info.

[BohdanBuinich]

@timg236 ok, I will try to reproduce on the new kernel but I can not understand why it works stable on the previous pieeprom-2024-01-05.bin

[peterharperuk]

I would guess this is network install causing problems as we look for an attached USB keyboard and it was enabled. Any chance you could set NET_INSTALL_ENABLED=0 in the bootloader config (sudo rpi-eeprom-config -e) to confirm?

[timg236]

@timg236 ok, I will try to reproduce on the new kernel but I can not understand why it works stable on the previous pieeprom-2024-01-05.bin

It looks like a non-standard kernel so would need the problem to reproducible on RPi OS

[BohdanBuinich]

It looks like a non-standard kernel so would need the problem to reproducible on RPi OS

I also have the same problem during runtime on RPi OS Linux raspberrypi 6.1.0-rpi7-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64

[  276.509841] EXT4-fs error (device sda2): ext4_journal_check_start:83: comm systemd-journal: Detected aborted journal
[  276.509874] EXT4-fs error (device sda2): ext4_journal_check_start:83: comm apt-get: Detected aborted journal
[  276.511710] EXT4-fs error (device sda2): ext4_journal_check_start:83: comm kworker/u8:2: Detected aborted journal
[  276.511739] EXT4-fs (sda2): Remounting filesystem read-only
[  276.511742] EXT4-fs (sda2): ext4_writepages: jbd2_start: 256 pages, ino 231673; err -30
dpkg: unrecoverable fatal error, aborting:
 unable to install updated status of 'vlc-data': Read-only file system
E: Sub-process /usr/bin/dpkg returned an error code (2)
W: Problem unlinking the file /var/cache/apt/pkgcache.bin - pkgDPkgPM::Go (30: Read-only file system)
rpi@raspberrypi:~$ [  276.787769] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #226854: comm cleanup: reading directory lblock 0
[  276.799254] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #226854: comm cleanup: reading directory lblock 0
[  279.007054] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #2: comm packagekitd: reading directory lblock 0
[  279.017943] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #226998: comm packagekitd: reading directory lblock 0
[  279.029252] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #226998: comm packagekitd: reading directory lblock 0
[  284.920704] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #227104: comm cron: reading directory lblock 0
[  284.931484] EXT4-fs error (device sda2): __ext4_find_entry:1682: inode #227104: comm cron: reading directory lblock 0

@peterharperuk I have the same problem with NET_INSTALL_ENABLED=0

################################################################################
[all]
BOOT_UART=1
POWER_OFF_ON_HALT=0
BOOT_ORDER=0xf461
NET_INSTALL_ENABLED=0
################################################################################

[peterharperuk]

Any chance you could try pieeprom-2024-01-15.bin? something like... sudo rpi-eeprom-update -f /lib/firmware/raspberrypi/bootloader-2712/latest/pieeprom-2024-01-15.bin

[BohdanBuinich]

Any chance you could try pieeprom-2024-01-15.bin? something like... sudo rpi-eeprom-update -f /lib/firmware/raspberrypi/bootloader-2712/latest/pieeprom-2024-01-15.bin

after upgrading and downgrading to a different version of eeprom I can't reproduce the problem (very strange) I will try to run additional tests today, and if nothing happens, I will close the problem thanks for support

[C0rn3j]

@BohdanBuinich You have the fabled RTL9210 chip with its usual 30 issues and some more.

The main one I can see you're hitting is UAS mode being enabled and IO is triggering the enclosure to fail and disk goes RO, that you can fix with adding the device to quirks.
This has been blacklisted in Linux already but will take some time to propagate to Pi OS in a later kernel release.

Update firmware of your enclosure, ensure you're using a short cable, yada yada.

See my comment here (I was not able to resolve my issues, yet, I can't even get it to boot in the first place) https://github.com/raspberrypi/linux/issues/4130#issuecomment-1925489269 and my own entry here describing all of the things you're possibly hitting https://rys.rs/books/wiki/page/raspberry-pi-5-nvme-and-usb-woes

My next trip is USB_MSD_PWR_OFF_TIME=0 when I've the time, if that's even a thing on the Pi5 anymore.
EDIT: Wow, that actually helped, got it to boot.

[actraiser30]

hi,another thing guys if possible,i have an rtl9201 and when i restart the Raspberry pi the hdd disk power off and power on again is it possible to not having that??

not to power off the disk at all on restart or shut down??

[timg236]

hi,another thing guys if possible,i have an rtl9201 and when i restart the Raspberry pi the hdd disk power off and power on again is it possible to not having that??

not to power off the disk at all on restart or shut down??

No, VBUS is always power off across a reboot and USB_MSD_PWR_OFF_TIME does nothing on Pi5

[actraiser30]

I have the 5v power rail on the usb 3 cable that is connected on the raspberry pi 4 not 5 disabled by using the tape mod,so the hard drive takes power from external power supply with an y cable,so it should not do that.any suggestions???

[lurch]

What does "the tape mod" mean?

[actraiser30]

You put a small PVC tape on the 5v power pin in the USB cable to cut the power.