raspberrypi / firmware

This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware.
5.17k stars 1.68k forks source link

Cannot recover the rpi 4 bootloader when it was previously configured to pxe boot #1310

Open rgl opened 4 years ago

rgl commented 4 years ago

When the rpi 4 bootloader was previously configured (with /lib/firmware/raspberrypi/bootloader/beta/pieeprom-2019-11-18.bin) to only boot from the network as:

BOOT_ORDER=0x2
NET_BOOT_MAX_RETRIES=-1

It cannot be recovered using the recovery procedure (BTW, this should be available as a .img file as the normal raspbian image, which would allow us to use the same procedure, e.g., using balena etcher).

pelwell commented 4 years ago

Please explain how the recovery procedure does not work.

The recovery image only consists of a few files, so publishing them as an archive is more flexible. An image file requires a spare card, but the archive can be extracted onto an existing card without breaking it.

timg236 commented 4 years ago

That doesn't sound right. The rescue image (recovery.bin) doesn't read from the EEPROM config or have code for boot-order or retries. It's actually separate code in order to keep it as small and as simple as possible.

I've just tried pieeprom-2019-11-18 with only those two lines modified. As expected it tries network boot forever. The rescue image from the downloads page on a blank FAT32 sd-card works for me.

It might be worth checking the hashes of the files on your rescue sd-card and against those extracted from the .zip e.g. perhaps pieeprom.bin already existed and was read-only ?

I think there's an internal feature request for a .img file in addition to the .zip because of limited tool support for reformatting a large exFAT sd-card as FAT on Windows. Although, this tool is pretty good http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm

Edit: If the EEPROM image is called pieeprom.bin then recovery.bin always sends debug output to the UART because it's assumed that this is a rescue rather than a silent update (pieeprom.upd)

rgl commented 4 years ago

I've unzipped the recovery zip contents to an empty FAT32 sdcard but when I rebooted the pi it still booted from the network. It should have read recovery.bin from the sdcard and reflashed eeprom with the defaults correct? like BOOT_ORDER set to 0x1 etc.

timg236 commented 4 years ago

Sorry, we can't tell any more with UART logs. The above steps worked for me.

1) Make sure you wait at least 10 seconds for the rescue image to run and that it produces a stable flashing green LED for another 10 seconds (probably overkill) 2) Verify that the Raspbian sd-card does NOT have a recovery.bin file on any of the FAT partitions. 3) Validate the checksums of the rescue image files

The recovery.bin file just writes a raw binary blob to the SPI EEPROM and does not attempt to parse the contents. It just checks that the size of the image is correct and the the hash matches the .sig.

rgl commented 4 years ago

I could only recover it by doing a pxe boot, reflash as normally one does with rpi-eeprom-update -d -f pieeprom-netboot.bin and with a normal raspbian sdcard in the pi. It did not work with just having the recovery files in a empty FAT32 sdcard.

timg236 commented 4 years ago

There's nothing so far to suggest that the rescue image is affected by the EEPROM configuration. Instead, for some reason related to the sd-card/formatting or board it isn't working for you.

I suggest that you try 1) Installing Raspbian on the sd-card which you used for the rescue disk 2) Verifying the activity LED behaviour during the rescue image. 3) Trying the rescue image on a different sd-card e.g. unzip the files to the boot partition on the Raspbian sd-card. 4) Use the rescue image to update the bootloader from the current booting state. e.g. add a comment (lines starting with #) and verify that the comment is visible via bootloader_config

Faeranne commented 3 years ago

Wanted to add to this, because I am having a similar issue. I've collected logs, which I am appending as a pastebin link.

https://pastebin.com/N1ctJSKJ

The logs loop like that, never acknowledging the card was connected. I've tried multiple cards of multiple sizes, and have more than one pi in this state. As far as I can tell, if BOOT_ORDER doesn't have 1 somewhere in it, the pi will never check the SD card, and thus never find the recovery.bin. If I'm right about this, there isn't a way to fix this just with an SD card.

As a workaround, as long as you can successfully setup a TFTP server that the pi can eventually reach, you can put the original recovery.bin from the recovery zip there, and it will fix the firmware.

Faeranne commented 3 years ago

Ok, slight addendum to my above comment. You actually just need to copy pieeprom.bin and pieeprom.sig to the TFTP server, but you do have to rename pieerom.bin to pieeprom.upd.