fomu can be be bricked by flashing too large of an image

royragsdale commented 2 years ago

Summary

Issue: Flashing a large file (4MB) with dfu-utilwill brick fomu.

Recommendation: Cut a new release of foboot.

This issue has already been fixed in master (e76d5f96c4cb7766b344168f342ac9c93960a6eb), but the fix is not included in the current release (v2.0.3).

Details

I encountered this when I built a version of micropython with a different toolchain that resulted in an erroneously large binary. Upon flashing my fomu became unresponsive (e.g. no lights, no dfu).

The following describes reproducing the issue to confirm:

Before

Tested using fomu-pvt running foboot v2.0.2 (v2.0.3 does not have any changes that would impact this behavior).

[Nov24 09:58] usb 2-1: new full-speed USB device number 7 using xhci_hcd
[  +0.151663] usb 2-1: New USB device found, idVendor=1209, idProduct=5bf0, bcdDevice= 1.01
[  +0.000012] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[  +0.000005] usb 2-1: Product: Fomu PVT running DFU Bootloader v2.0.2
[  +0.000003] usb 2-1: Manufacturer: Foosn

$ dfu-util -l
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Found DFU: [1209:5bf0] ver=0101, devnum=7, cfg=1, intf=0, path="2-1", alt=0, name="Fomu PVT running DFU Bootloader v2.0.2", serial="UNKNOWN"

Build and flash a BAD binary (too large)

script to create a patterned flash image, to allow easy identification of offsets.

#!/usr/bin/env python3

SZ = (4096 * 1024)//4

with open('BAD.flash-pattern.dfu', 'wb') as out:
    for b in range(SZ):
        out.write(b.to_bytes(4, byteorder='big'))

produces a 4MB file

$ ls -al BAD.flash-pattern.dfu
-rw-r--r-- 1 r r 4194304 Nov 24 10:03 BAD.flash-pattern.dfu

WARNING running this commands will BRICK your fomu. You will NOT be able to recover over usb.

$ dfu-util -D BAD.flash-pattern.dfu
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=========================] 100%      4194304 bytes
Download done.
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(8) = dfuMANIFEST-WAIT-RESET, status(0) = No error condition is present
Done!

fomu flashes red and is non-responsive to dfu-util.

After unplugging and replugging still non-responsive

[Nov24 10:07] usb 2-1: USB disconnect, device number 7
[Nov24 10:08] usb 2-1: new full-speed USB device number 8 using xhci_hcd
[  +0.127932] usb 2-1: device descriptor read/64, error -71
[  +0.236052] usb 2-1: device descriptor read/64, error -71
[  +0.236020] usb 2-1: new full-speed USB device number 9 using xhci_hcd
[  +0.127964] usb 2-1: device descriptor read/64, error -71
[  +0.236062] usb 2-1: device descriptor read/64, error -71
[  +0.108050] usb usb2-port1: attempt power cycle
[  +0.411921] usb 2-1: new full-speed USB device number 10 using xhci_hcd
[  +0.000293] usb 2-1: Device not responding to setup address.
[  +0.207896] usb 2-1: Device not responding to setup address.
[  +0.207805] usb 2-1: device not accepting address 10, error -71
[  +0.127910] usb 2-1: new full-speed USB device number 11 using xhci_hcd
[  +0.000152] usb 2-1: Device not responding to setup address.
[  +0.208152] usb 2-1: Device not responding to setup address.
[  +0.207762] usb 2-1: device not accepting address 11, error -71
[  +0.000054] usb usb2-port1: unable to enumerate USB device

When connecting directly to the SPI and using fomu flash, you can confirm that the flash "wrapped" around and clobbered the low bytes of the SPI. The resulting overwrite corresponds to offset 0x003c0000 in BAD.flash-pattern.dfu

$ sudo ./fomu-flash/fomu-flash -p 0
00000000 00 0f 00 00 00 0f 00 01  00 0f 00 02 00 0f 00 03  |................|
00000010 00 0f 00 04 00 0f 00 05  00 0f 00 06 00 0f 00 07  |................|
00000020 00 0f 00 08 00 0f 00 09  00 0f 00 0a 00 0f 00 0b  |................|

Expected Behavior

When using a foboot build from 882ce04da8b97084ef95f5a967a0e2b7312b19c2 the expected behavior occurs and you are prevented from flashing (using dfu-util) a payload that is too large

$ dfu-util -l
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Found DFU: [1209:5bf0] ver=0101, devnum=29, cfg=1, intf=0, path="2-1", alt=0, name="Fomu PVT running DFU Bootloader v2.0.3-57-g882ce04", serial="UNKNOWN"

$ dfu-util -D BAD.flash-pattern.dfu
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=======                  ]  30%      1294336 bytes failed!
state(10) = dfuERROR, status(8) = Cannot program memory due to received address that is out of range

fomu continues to operate as usual.

mithro commented 2 years ago

FYI - @xobs

xobs commented 2 years ago

Can you please see if v2.0.4 (https://github.com/im-tomu/foboot/tree/master/releases/v2.0.4) solves the issue?

royragsdale commented 2 years ago

looks like only the .h files were pushed. want me to build locally and test?

xobs commented 2 years ago

My mistake. Normally .elf and .dfu files aren't included, and I forgot to add them with -f. Please try again.

Note that I needed to manually patch the SPI bus for the Hacker version of the board, due to a refactor in the spi_core that assumes all SPI devices have four wires even if they are only dual SPI. I believe this was fixed later on, but litex has made many changes since foboot was released, and updating it might be involved.

royragsdale commented 2 years ago

Thank you! Tested on a pvt board (I don't have a hacker version). Works like a charm. I really appreciate you jumping on this.

Notes on what I tested:

flash

$ dfu-util -D pvt-updater-v2.0.4.dfu
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Match vendor ID from file: 1209
Match product ID from file: 70b1
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=========================] 100%       112916 bytes
Download done.
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(8) = dfuMANIFEST-WAIT-RESET, status(0) = No error condition is present
Done!

confirm

$ dfu-util -l
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Found DFU: [1209:5bf0] ver=0101, devnum=31, cfg=1, intf=0, path="2-1", alt=0, name="Fomu PVT running DFU Bootloader v2.0.4", serial="UNKNOWN"

flashing micropython from fomu-workshop works.

It properly rejects files that are too large.

$ dfu-util -D BAD.flash-pattern.dfu
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=======                  ]  30%      1290240 bytes failed!
state(10) = dfuERROR, status(8) = Cannot program memory due to received address that is out of range

xobs commented 2 years ago

Thanks for the very detailed reports!

royragsdale commented 2 years ago

Absolutely, happy to add. Fomu has been fantastic for learning. I really appreciate the project. Thank you also for the super responsive feedback.

For what it's worth, I see the v2.0.4 tag, but not a release under https://github.com/im-tomu/foboot/releases. I'm not sure how Github handles/generates those.

xobs commented 2 years ago

It's manual currently. I just released it there.

im-tomu / foboot