search

raspberrypi / usbboot

Raspberry Pi USB booting code, moved from tools repository
Apache License 2.0
913 stars 231 forks source link

RPIBOOT BOOT_ORDER (0x3) doesn't work on CM4 without nRPIBOOT pulled low #223

Closed drebbe-intrepid closed 3 months ago

drebbe-intrepid commented 3 months ago

Describe the bug

RPIBOOT BOOT_ORDER (0x3) doesn't work on CM4 without nRPIBOOT pulled low.

I'm attempting to try and recover a CM4 without taking my case apart to get at nRPIBOOT (purely software solution).

Following documentation: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#BOOT_ORDER we should be able to put 0x3 anywhere in the BOOT_ORDER to allow USB Boot.

Is what I'm trying to accomplish possible or is it a limitation of the SoC?

Steps to reproduce the behaviour

  1. clone repo
  2. cd recovery
  3. change boot.conf BOOT_ORDER to 0x3
  4. nRPIBOOT low and power cycle
  5. ./update-pieeprom.sh && sudo ../rpiboot -d .
  6. nRPIBOOT disabled and power cycle
  7. Notice USB never enumerates on the host

Device(s)

Raspberry Pi CM4

Compute Module IO board.

Custom neoVI PI. Based on Module IO PCB.

RPIBOOT logs

N/A

Kernel logs

N/A

Device UART logs

N/A. I'll have to hook this up if needed for further debugging.

drebbe-intrepid commented 3 months ago

I'm going to confirm this is an issue with the Compute Module IO board also in the next couple days.

timg236 commented 3 months ago

BOOT_ORDER=0x3 doesn't make sense standalone. It should always be driven via rpiboot and with the nRPIBOOT pulled low.

You can force nRPIBOOT without the jumper by erasing the EEPROM (e.g. flashrom) but realistically you need to make sure that the jumper is accessible.

drebbe-intrepid commented 3 months ago

BOOT_ORDER=0x3 doesn't make sense standalone. It should always be driven via rpiboot and with the nRPIBOOT pulled low.

You can force nRPIBOOT without the jumper by erasing the EEPROM (e.g. flashrom) but realistically you need to make sure that the jumper is accessible.

The problem is you can't use rpiboot without nPRIBOOT. I'm looking for a pure software solution for manufacturing purposes. If BOOT_ORDER=0x3 was never intended to be functional is it a bug in documentation?

timg236 commented 3 months ago

Possibly, it should be marked as deprecated.

drebbe-intrepid commented 3 months ago

Another thought, when the eMMC is new nPRIBOOT isn't needed AFAIK, is there a way to "restore" the eMMC back to this state? This is the state I'm looking to be in even if the eMMC is "bad" (distro doesn't boot for various reasons).

timg236 commented 3 months ago

The eMMC is blank but the SPI FLASH containing the bootloader is populated. The bootrom will failover to RPIBOOT if the SPI FLASH is corrupt / empty but in order to do that you'd need to boot the OS.

drebbe-intrepid commented 3 months ago

It would be useful to have a bootrom that would default to RPIBOOT if USB is plugged in and if not eMMC/SDCard. is this currently possible?

pelwell commented 3 months ago

There's a clue in the word "bootrom". If it doesn't work now, it never will.

drebbe-intrepid commented 3 months ago

According to the docs the Boot ROM is configured by the EEPROM configuration and not OTP here: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#otp-registers-on-non-bcm2712-devices

According to the docs here: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#first-stage-bootloader

It looks like I'm getting stuck in the second stage loader as its not properly executing BOOT_ORDER=0x3. I believe in order to continue debugging this I'll need to hook up UART and see if there is anything useful there. I'm also under the assumption all the bootloaders are proprietary and not open source.

timg236 commented 3 months ago

The Boot ROM is implemented in the metal layer of the SOC and is only configurable via a small number of OTP registers. None of these enable RPIBOOT automatically. Since it's extremely expensive to spin another bootrom mask I think it's safe to say that this is not a good enough reason to do that.

The "EEPROM" SPI flash bootloader is configured via flash configuration data including the BOOT_ORDER. USB device boot is somewhat special and whilst noted as boot-order 0x3 it's only ever worked in the mode where the ROM loaded the bootloader via USB device boot.

It's already possible to select alternate bootmodes via GPIOs e.g. Network, USB, PCIe, HTTP or even boot to a failsafe Linux partition with a Linux initramfs that exposed the reset of the disk as mass-storage (copy the mass-storage gadget code).

Feature request noted but since you would have to use nRPIBOOT to change the boot-order via recovery.bin you would have to expose that or another GPIO anyway so therefore I think this is NGTF but will check the docs to see if they could be clearer.

nbuchwitz commented 3 months ago

Why not just pull nrpiboot when vusb is present? Add a simple usb switch for the data lines if you need the soc usb port exposed during operation. We do this in our products and it works without a flaw.

drebbe-intrepid commented 3 months ago

The Boot ROM is implemented in the metal layer of the SOC and is only configurable via a small number of OTP registers. None of these enable RPIBOOT automatically. Since it's extremely expensive to spin another bootrom mask I think it's safe to say that this is not a good enough reason to do that.

The "EEPROM" SPI flash bootloader is configured via flash configuration data including the BOOT_ORDER. USB device boot is somewhat special and whilst noted as boot-order 0x3 it's only ever worked in the mode where the ROM loaded the bootloader via USB device boot.

It's already possible to select alternate bootmodes via GPIOs e.g. Network, USB, PCIe, HTTP or even boot to a failsafe Linux partition with a Linux initramfs that exposed the reset of the disk as mass-storage (copy the mass-storage gadget code).

Feature request noted but since you would have to use nRPIBOOT to change the boot-order via recovery.bin you would have to expose that or another GPIO anyway so therefore I think this is NGTF but will check the docs to see if they could be clearer.

Thank you for the explanation. It wasn't clear to me from the documents the boot ROM is special and it looks like the second stage bootloader just piggy backs off of that feature. It makes sense now that when 0x3 is enabled nothing else works because that second stage bootloader probably doesn't init anything. As a suggestion for the documents it would be nice to note this is a "limitation" of 0x3.

berndbenner commented 3 months ago

BOOT_ORDER 0x4 ( USB-MSD) will also not work on cm4 with a boot-Partition on the eMMC-Flash. The cm4 will always boot from eMMC-Flash not from a USB stick. ( raspi-config B2 or rpi-eeprom-config --edit triggered ).

timg236 commented 3 months ago

That's incorrect. If boot-order 0x4 is not working then it's an issue with that particular USB-MSD device. N.B The micro-USB must be unplugged otherwise the hub for the type-A sockets is disabled.

berndbenner commented 3 months ago

Sorry, but its correct!

I have a 64GB Samsung Flash Drive FIT Stick, booting well on all RPIs 3,4,5,zero-w2 with (3 and zero-w2 with bootcode.bin on sd-card). This stick will boot on a regular RPI4 without a problem. I have used this stick also to boot the virgin cm4 module and setup the eMMC-partitions. After setup the eMMC, I try to boot from the USB-device again, first with raspi-config (B2) and the installed bootcode in the SOC and then with a updated eeprom and rpi-eeprom-config --edit, without success.

root@raspi-sl-2:/home/pi# rpi-eeprom-config --edit
Updating bootloader EEPROM
 image: /lib/firmware/raspberrypi/bootloader-2711/default/pieeprom-2024-04-15.bin
config_src: blconfig device
config: /tmp/tmp0pb2btrb/boot.conf
################################################################################
[all]
BOOT_UART=1
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=1
USB_MSD_STARTUP_DELAY=1000

[all]
BOOT_ORDER=0xf14

################################################################################

*** To cancel this update run 'sudo rpi-eeprom-update -r' ***

*** CREATED UPDATE /tmp/tmp0pb2btrb/pieeprom.upd  ***

   CURRENT: Mon Apr 15 01:12:14 PM UTC 2024 (1713186734)
    UPDATE: Mon Apr 15 01:12:14 PM UTC 2024 (1713186734)
    BOOTFS: /boot/firmware
'/tmp/tmp.9cNYs0mH36' -> '/boot/firmware/pieeprom.upd'

UPDATING bootloader. This could take up to a minute. Please wait

*** Do not disconnect the power until the update is complete ***

If a problem occurs then the Raspberry Pi Imager may be used to create
a bootloader rescue SD card image which restores the default bootloader image.

flashrom -p linux_spi:dev=/dev/spidev0.0,spispeed=16000 -w /boot/firmware/pieeprom.upd
UPDATE SUCCESSFUL
root@raspi-sl-2:/home/pi#

If I wipe the partition table of the eMMC with dd the bootcode in the cm4 will hang in a endless loop reporting 

Boot mode: RESTART (0f) order 0
Boot mode USB-MSD (04) order f1
USB xHC init failed
Boot mode: SD (01) order f
Trying partition: 0
Trying partition: 0

After restoring the partition table on the eMMC with dd from a file the cm4 boot from eMMC and the USB-Stick ia accessable.

There must be a USB 2.0 initialization problem in the bootcode on cm4-hardware.

timg236 commented 3 months ago

I'm closing this issue because the original questions regarding USB device boot (0x3) was resolved.

The logs indicate that the XHCI init failed on that bootloader release - if you are seeing XHCI issues with boot-order 0xf14 please raise a firmware issue with logs and any other information that you can provide about the hardware setup. It would also be worth trying an older version of the bootloader e.g. something from 2023 in case it's a regression in more recent releases.

Edit: It would also be worth trying the pieeprom-2024-05-17 release based on the common 2712/2711 firmware branch - in the LATEST directory.

timg236 commented 3 months ago

@berndbenner I spotted that you are using 0x4 for USB mass-storage but that would only work for a PCIe XHCI card on CM4, or on board VLI chip on a Pi4/Pi400.

For the CM4 SOC XHCI controller you need to use 0x5 https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#BOOT_ORDER

This is explains why you see it trying to initialise the host-controller, failing (because there isn't a PCIe one) then switching to SD/MMC.

drebbe-intrepid commented 3 months ago

@timg236 I wouldn't consider this closed as its a hole in documentation IMHO. Without the explanation here it wouldn't have been clear.