Urboot 8.0 pre-release for testing

[stefanrueger]

• Generally shorter(*) bootloaders given the same features
• Added new feature U which updates flash, ie, only changes flash when new content is different (less wear and faster)
• urboot.hex repository will have now has v8.0 bootloaders (git willing)

Now they need a bit of testing...

(*) The way urboot is compiled tends to fill out unused space with frills so that shorter code isn't readily visible, but it is

[mokrafoka]

Pleasant to see that the new version uses even less code. Especially now it seems to fit into two pages even with reset protection. I think this is the reason why it's enabled by default now, right? For me its like:

make MCU=atmega328p F_CPU=8000000L NAME=mega328_autobaud_LedB0_8MHz AUTOBAUD=1 BLINK=1 LED=AtmelPB0 LEDPOLARITY=1 EEPROM=0 URPROTOCOL=1 VBL=1 PROTECTRESET=1
...
-- 262 384 u7.7 w-u-jPra- mega328_autobaud_LedB0_DualInt_8MHz.hex

For the v7.x version, and

make MCU=atmega328p F_CPU=8000000L NAME=mega328_autobaud_LedB0_8MHz AUTOBAUD=1 BLINK=1 LED=AtmelPB0 LEDPOLARITY=1 EEPROM=0 URPROTOCOL=1 VBL=1
...
-- 254 256 u8.0 w-u-jPra- mega328_autobaud_LedB0_DualInt_8MHz.hex

For the v8 version, where the protect-reset is already enabled by default. It's a tight fit... but still it fits and gets even smaller if protect-reset is disabled. ... And even dual boot fits in 3 pages now ;-)

However I one issue, that was already partially present in the v7 version as well, gives me some head aches.

-#if VBL == VBL_PATCH || VBL == VBL_VERIFY
-             vbl_patch();

The vbl_patch() code has been now completely removed from dual_boot() function. I don't get it. IMHO this should be enabled in the dual_boot case if the boot loader uses VBL. If you use the avrdude/urclock then the VBL gets patched by avrdude automatically. But in case of OTA this magic help isn't available. One would have to patch the new app image locally before uploading it to the target. I doubt this will end up in clean production process. Thus I think the code should look similar to this:

diff --git a/src/urboot.c b/src/urboot.c
index b00be1f..b2870d0 100644
--- a/src/urboot.c
+++ b/src/urboot.c
@@ -1360,7 +1360,7 @@ typedef uint32_t VBLvect_t;     // Larger AVRs have 4-byte vectors (jmp)
 #define appVectOrigHi (((uint16_t *) & appVectOrig)[1])

-#if VBL >= VBL_VERIFY || VBL == VBL_PATCH
+#if VBL >= VBL_VERIFY || VBL == VBL_PATCH || ( defined(DUAL) && VBL != 0 )

 // Check VBL_VECT_NUM vector is on the first page (patching and verifying code assume that)
 #if VBL_VECT_NUM >= SPM_PAGESIZE/(FLASHin8k? 2: 4)

To enable the void vbl_patch() and to execute the patch function out of dual_boot() as well if the boot loader is compiled with VBL support. This will allow you to (OTA) upload any application image, which in turn would be of universal use: either for OTA or avrdude/urclock upload process.

That said, Thanks for the impressive work! \o/ (I think this is the best avr project I've seen since I work with those small beasts, and this are more than 15 years now...)

[stefanrueger]

git willing

It took my PC 23 hours to compile the 1m+ urboot bootloaders, and git add the best part of 24 hours to process them for the urboot.hex repository. So, now that the v8.0 precompiled bootloaders are there I was wondering if you could give them a spin @MCUdude and @mcuee. I tried m328p and m1284p, but I would be interested in how the t2313 (EEPROM code in 256 bytes, without EEPROM 192 bytes) and the t13 fare with the beta version. I found optimisations specifically for these small devices but couldn't test them...

And then there is the new U update feature. Best to try that with the larger bootloaders and larger parts, eg, m2560.

... And even dual boot fits in 3 pages now ;-)

@mokrafoka Ahh, good. I had not realised. I always seem to compile EEPROM in for dual boot.

The vbl_patch() code has been now completely removed from dual_boot() function. I don't get it.

Yes, patching is best done outside the bootloader except for reset protection, which is important to protect bootloaders from bricking. BTW, reset protection is by default added iff there is space left for it; that hasn't changed; bootloader protection from overwriting itself (PROTECTME) has been on and can no longer be switched off from v7.7; that hasn't changed either.

Note that patching the application jump costs a lot of code: The bootloader would have to check whether it the sketch is already patched; if so don't do it again (that would lose the jmp to the application). The bootloader would need to figure out where the application sits (Is it a relative jump? Is it an absolute jump?) Then copy the jmp to the correct vector; if it's a relative jmp shorten the distance to the application accordingly. These things cost a lot of code and actually are best done by a little PC-application that process the output of the compiler. You could even use AVRDUDE for that:

avrdude -c urclock -p m328p -U mysketch.hex -T "save flash 0 -385 patched-sketch.hex:i"

This assumes a 384-byte bootloader. You'd need the part with the correct bootloader on, though. And it would cost the time to burn the flash. Once could imagine a variant of the dryrun programmer that emulates urclock, eg,

avrdude -c dryrun -p m328p -x urclock -U urboot_bootloader.hex -U mysketch.hex -T "save flash 0 -385 patched.hex

The latter doesn't exist (yet) but would be much much better than ballooning the bootloader for every part.

[mcuee]

Nice improvements. I will carry out some tests over the weekend.

[stefanrueger]

I have done some testing for five boards in discussions #36 (Digispark ATtiny85, Digispark Pro ATtiny167, Mega R3 ATmega2560, Pro Mini ATmega328P, UrclockMega ATmega1284P). @MCUdude, @mcuee Feel free to add tests for your own boards that you happen to have if they are different MCUs. This does not need to be the full UPDATE_FL=4 level; it is sufficient to test any bootloaders types you usually deploy. You can select from v8.0 precompiled bootloaders.

[MCUdude]

I have limited time in the following days, but I'll try on a few different MCUs. Urboot 8.0 will require Avrdude 8.0? If so, I won't be updating the bootloaders in my Arduino cores any time soon because they rely on Arduino's statically built Avrdude binaries.

[stefanrueger]

Urboot 8.0 will require Avrdude 8.0?

No. Urboot 8.0 bootloaders work with Avrdude 7.1 onwards, the earliest AVRDUDE version that knew urboot bootloaders.

The only difference is that older avrdude -xshowall displays urboot 8.0's new update feature as s/u instead of u/U shown by git main (and upcoming Avrudude 8.0). Luckily, this bit in the version capability byte is for user info only and not relied upon by AVRDUDE. I would have liked a better feature letter than U for the new update feature, but couldn't come up with a good mnemonic for the free letters (b g i k l m n o q t x y z). Suggestions welcome!

Note that the test-avrdude command needs AVRDUDE 7.3, though, as it relies on the overhauled terminal part command.

[mcuee]

First test with ATmega328PB -- good.

$ ./build_mingw64_nt-10.0-22631/src/avrdude.exe -c urclock -P ch340 -p m328pb -xshowall -qq
ffffffffffff 2024-02-25 14.05 rjmp_loops_for_bootloaders_32768B.hex 24576 store 7759 meta 49 boot 384 u8.0 weU-jPrac vector 25 (SPM_Ready) ATmega328PB

$ ./tools/test-avrdude -e ./build_mingw64_nt-10.0-22631/src/avrdude.exe -p "-c urclock -P ch340 -p m328pb"
Testing ./build_mingw64_nt-10.0-22631/src/avrdude.exe version 7.3-20240601 (2754cb0d)
Prepare "-c urclock -P ch340 -p m328pb" and press 'enter' or 'space' to continue. Press any other key to skip
✅   1.044 s: chip erase
✅   2.504 s: flash -U write/verify holes_rjmp_loops_32768B.hex
✅   1.119 s: flash -T write/verify holes_rjmp_loops_32768B.hex
✅   0.502 s: eeprom check whether programmer can flip 0s to 1s
✅   1.830 s: eeprom -U write/verify holes_pack_my_box_1024B.hex
✅   3.484 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_1024B.hex
✅   1.715 s: chip erase and spot check flash is actually erased

[mcuee]

I have limited time in the following days, but I'll try on a few different MCUs. Urboot 8.0 will require Avrdude 8.0? If so, I won't be updating the bootloaders in my Arduino cores any time soon because they rely on Arduino's statically built Avrdude binaries.

Hmm, our avrdude repo should have the statically linked avrdude binaries for avrdude 7.3 for both Windows and Linux (including libserialport support). I did not put up macOS binaries because I could not build libserialport in the cross-compile container. I can upload hat macOS binary as well if you want -- it will be no worse than Arduino avrdude-packing provided 7.2 binaries. We can do the same for upcoming avrdude 8.0 release. https://github.com/avrdudes/avrdude/releases/tag/v7.3 https://github.com/arduino/avrdude-packing/releases/tag/7.2-arduino.1

Reference:

• https://github.com/avrdudes/avrdude/issues/1679
• https://github.com/arduino/avrdude-packing/issues/31

[mcuee]

Hmm, I spent some time this afternoon but I could not get either ATmega32A and ATtiny13A to work. It has nothing to do with urboot but rather I have issues with the USB to TTL converter connections to the boards. I may have to try another time.

[MCUdude]

Hmm, I spent some time this afternoon but I could not get either ATmega32A and ATtiny13A to work. It has nothing to do with urboot but rather I have issues with the USB to TTL converter connections to the boards. I may have to try another time.

Stefan just ordered a few boards from me, so he will have reliable hardware to test with in just a few days.

Hmm, our avrdude repo should have the statically linked avrdude binaries for avrdude 7.3 for both Windows and Linux (including libserialport support)

@mcuee it would be really neat if we could provide statically built binaries for these platforms. And even better if we used the patched version of libserialport that also works on the latest macOS versions.

{
          "name": "avrdude",
          "version": "7.2-arduino.1",
          "systems": [
            {
              "size": "1243922",
              "checksum": "SHA-256:b5b88e4f52c0edb861f5ae54acbd752f9ecd562a4db4d821ab8c276ee5696dab",
              "host": "arm-linux-gnueabihf",
              "archiveFileName": "avrdude_7.2-arduino.1_Linux_ARMv6.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_Linux_ARMv6.tar.gz"
            },
            {
              "size": "1348250",
              "checksum": "SHA-256:a3862e6a38668c2688dc0822b16c3f824612cc606259b6813bc5778d36c92ba9",
              "host": "aarch64-linux-gnu",
              "archiveFileName": "avrdude_7.2-arduino.1_Linux_ARM64.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_Linux_ARM64.tar.gz"
            },
            {
              "size": "928946",
              "checksum": "SHA-256:256cbde856714a18c11b4c99f6b00eed65e4208c92425fa9cfd75a5bdfb6ab09",
              "host": "x86_64-apple-darwin12",
              "archiveFileName": "avrdude_7.2-arduino.1_macOS_64bit.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_macOS_64bit.tar.gz"
            },
            {
              "size": "1256881",
              "checksum": "SHA-256:25e1b568757d9a58b9663e4493ffc04b9e6d690535c9c1e6c1db7d1ecffb5eff",
              "host": "x86_64-linux-gnu",
              "archiveFileName": "avrdude_7.2-arduino.1_Linux_64bit.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_Linux_64bit.tar.gz"
            },
            {
              "size": "1252284",
              "checksum": "SHA-256:86a811f6ba2bebbb717a524fcff495f0ebb146abcd15d341a10d867b8e8c83fd",
              "host": "i686-linux-gnu",
              "archiveFileName": "avrdude_7.2-arduino.1_Linux_32bit.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_Linux_32bit.tar.gz"
            },
            {
              "size": "1682746",
              "checksum": "SHA-256:5536c9fcb41f4a36aa55b3711a0d74943a401261b15cc8aae2473c30ff292021",
              "host": "i686-mingw32",
              "archiveFileName": "avrdude_7.2-arduino.1_Windows_32bit.tar.gz",
              "url": "http://downloads.arduino.cc/tools/avrdude_7.2-arduino.1_Windows_32bit.tar.gz"
            }

[stefanrueger]

just ordered a few boards from me, so he will have reliable hardware to test with in just a few days.

... covering m32a and t13a. And I ordered dev hardware from an aliexpress vendor for the t2313.

@MCUdude @mcuee Testing any other (popular or exotic) classic MCUs you might have access to highly appreciated. @mcuee, thanks for testing the m328pb!

[MCUdude]

I'm using the most "maxed out" autobaud vector bootloader option for all targets. Tested with the latest Avrdude git main. This looks very promising!

$ ./test-avrdude -p "-curclock -patmega16 -P /dev/cu.usbserial-1410 -b38400" 
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega16 -P /dev/cu.usbserial-1410 -b38400" and press 'enter' or 'space' to continue. Press any other key to skip
✅   1.001 s: chip erase
✅   3.030 s: flash -U write/verify holes_rjmp_loops_16384B.hex
✅   1.273 s: flash -T write/verify holes_rjmp_loops_16384B.hex
✅   0.511 s: eeprom check whether programmer can flip 0s to 1s
✅   1.858 s: eeprom -U write/verify holes_pack_my_box_512B.hex
✅   3.486 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_512B.hex
✅   1.824 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -patmega324p -P /dev/cu.usbserial-1410 -b125000" 
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega324p -P /dev/cu.usbserial-1410 -b125000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   1.690 s: chip erase
✅   3.172 s: flash -U write/verify holes_rjmp_loops_32768B.hex
✅   1.004 s: flash -T write/verify holes_rjmp_loops_32768B.hex
✅   0.449 s: eeprom check whether programmer can flip 0s to 1s
✅   1.725 s: eeprom -U write/verify holes_pack_my_box_1024B.hex
✅   3.283 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_1024B.hex
✅   2.114 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -patmega8535 -P /dev/cu.usbserial-1410 -b1000000" 
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega8535 -P /dev/cu.usbserial-1410 -b1000000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.937 s: chip erase
✅   1.281 s: flash -U write/verify holes_rjmp_loops_8192B.hex
✅   0.478 s: flash -T write/verify holes_rjmp_loops_8192B.hex
✅   0.692 s: eeprom check whether programmer can flip 0s to 1s
✅   1.528 s: eeprom -U write/verify holes_pack_my_box_512B.hex
✅   2.672 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_512B.hex
✅   1.032 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -patmega165p -P/dev/cu.usbserial-1410 -b250000" -v
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega165p -P/dev/cu.usbserial-1410 -b250000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.971 s: chip erase
✅   1.539 s: flash -U write/verify holes_rjmp_loops_16384B.hex
✅   0.591 s: flash -T write/verify holes_rjmp_loops_16384B.hex
✅   0.448 s: eeprom check whether programmer can flip 0s to 1s
✅   1.036 s: eeprom -U write/verify holes_pack_my_box_512B.hex
✅   1.767 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_512B.hex
✅   1.136 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -patmega2561 -P/dev/cu.usbserial-1410 -b1000000"
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega2561 -P/dev/cu.usbserial-1410 -b1000000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   2.106 s: chip erase
✅   3.155 s: flash -U write/verify holes_rjmp_loops_262144B.hex
✅   1.253 s: flash -T write/verify holes_rjmp_loops_262144B.hex
✅   0.485 s: eeprom check whether programmer can flip 0s to 1s
✅   5.008 s: eeprom -U write/verify holes_pack_my_box_4096B.hex
✅   9.099 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_4096B.hex
✅   2.099 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -patmega64 -P/dev/cu.usbserial-1410 -b500000"
Testing avrdude version 7.3-20240222
Prepare "-curclock -patmega64 -P/dev/cu.usbserial-1410 -b500000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.463 s: chip erase
✅   1.396 s: flash -U write/verify holes_rjmp_loops_65536B.hex
✅   0.741 s: flash -T write/verify holes_rjmp_loops_65536B.hex
✅   0.556 s: eeprom check whether programmer can flip 0s to 1s
✅   5.085 s: eeprom -U write/verify holes_pack_my_box_2048B.hex
✅   9.908 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_2048B.hex
✅   0.971 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -pat90can128 -P/dev/cu.usbserial-1410 -b115200"
Testing avrdude version 7.3-20240222
Prepare "-curclock -pat90can128 -P/dev/cu.usbserial-1410 -b115200" and press 'enter' or 'space' to continue. Press any other key to skip
✅   1.630 s: chip erase
✅   5.532 s: flash -U write/verify holes_rjmp_loops_131072B.hex
✅   2.590 s: flash -T write/verify holes_rjmp_loops_131072B.hex
✅   0.670 s: eeprom check whether programmer can flip 0s to 1s
✅  10.481 s: eeprom -U write/verify holes_pack_my_box_4096B.hex
✅  21.043 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_4096B.hex
✅   3.044 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -pattiny1634 -P/dev/cu.usbserial-1410 -b500000"
Testing avrdude version 7.3-20240222
Prepare "-curclock -pattiny1634 -P/dev/cu.usbserial-1410 -b500000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.963 s: chip erase
✅   2.040 s: flash -U write/verify holes_rjmp_loops_16384B.hex
✅   0.811 s: flash -T write/verify holes_rjmp_loops_16384B.hex
✅   0.686 s: eeprom check whether programmer can flip 0s to 1s
✅   0.851 s: eeprom -U write/verify holes_pack_my_box_256B.hex
✅   1.068 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_256B.hex
✅   1.220 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -pattiny841 -P/dev/cu.usbserial-1410 -b57600"
Testing avrdude version 7.3-20240222
Prepare "-curclock -pattiny841 -P/dev/cu.usbserial-1410 -b57600" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.948 s: chip erase
✅   2.666 s: flash -U write/verify holes_rjmp_loops_8192B.hex
✅   0.975 s: flash -T write/verify holes_rjmp_loops_8192B.hex
✅   0.594 s: eeprom check whether programmer can flip 0s to 1s
✅   1.139 s: eeprom -U write/verify holes_pack_my_box_512B.hex
✅   1.991 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_512B.hex
✅   1.511 s: chip erase and spot check flash is actually erased

$ ./test-avrdude -p "-curclock -pattiny4313 -P /dev/cu.usbserial-00FEBF3C -b500000"
Testing avrdude version 7.3-20240222
Prepare "-curclock -pattiny4313 -P /dev/cu.usbserial-00FEBF3C -b500000" and press 'enter' or 'space' to continue. Press any other key to skip
✅   0.722 s: chip erase
✅   0.828 s: flash -U write/verify holes_rjmp_loops_4096B.hex
✅   0.435 s: flash -T write/verify holes_rjmp_loops_4096B.hex
✅   0.429 s: eeprom check whether programmer can flip 0s to 1s
✅   0.643 s: eeprom -U write/verify holes_pack_my_box_256B.hex
✅   0.907 s: eeprom -T write/verify holes_{the_five_boxing_wizards,pack_my_box}_256B.hex
✅   0.684 s: chip erase and spot check flash is actually erased

[stefanrueger]

@MCUdude This is brilliant news! Looks like I can release u8.0 soon. Thank you so much