Request: documentation of firmware format

[djphoenix]

Hi there. I trying to implement firmware generation without esptool2 binary. Now I stuck in firmware header format, so, if you can provide well-documented headers description (and so procedure of checksum generation), it will be great point.

So "hex2" header are already done with:

$(RBOOT_BUILD_BASE)/rboot-hex2a.h: $(RBOOT_BUILD_BASE)/rboot-stage2a.elf
  @ mkdir -p $(dir $@)
  @ rm -f $@
  @ $(OBJDUMP) -f $< | grep 'start address' | awk '{print "const uint32 entry_addr = " $$3 ";"}' >> $@
  @ $(OBJDUMP) -j .text -h $< | grep ' .text ' | awk '{print "const uint32 _text_addr = 0x" $$4 ";"}' >> $@
  @ $(OBJDUMP) -j .text -h $< | grep ' .text ' | awk '{print "const uint32 _text_len  = 0x" $$3 ";"}' >> $@
  @ echo 'const uint8  _text_data[] = {' >> $@
  @ $(OBJCOPY) --dump-section .text=/dev/stdout $< | xxd -i >> $@
  @ echo '};' >> $@

I will make PR for all no-esptool2-related stuff after done.

[raburton]

rBoot uses the same rom format as the sdk bootloader (v1.2+), so you can create them in the same way as various tools out there that do that. I also wrote about the format some time ago here: http://richard.burtons.org/2015/05/17/decompiling-the-esp8266-boot-loader-v1-3b3/ (when I originally wrote esptool2). Esptool2 is of course is open source, so you can have a look at the code that creates the rom if that helps. The only difference is that optionally the checksum can be extended to include the irom section as well at the iram section, but the algorithm is the same (just a simple xor).

[djphoenix]

Okay, now is something strange...

I done this:

$ esptool2 -quiet -bin -boot2 -iromchksum ../img/firmware.out firmware.bin .text .data
$ esptool.py elf2image -o firmware2.bin -e 2 ../img/firmware.out 
$ xxd -ps firmware.bin > firmware.hex
$ xxd -ps firmware2.bin > firmware2.hex
$ diff firmware.hex firmware2.hex 
15309c15309
< 000571fa083112c1100df0000080fe3ff00e000025732025750a00006d61
---
> 000571fa083112c1100df000e085fe3ff00e000025732025750a00006d61
15437c15437
< 0100000000000034
---
> 01000000000000b1

How I see, firmware checksums at end are differs (because iromchksum, and yes, it is correct when I not pass this option to esptool2). But also different in .data address (3ffe8000 with 3ffe85e0). And this is strange miss that I can't describe for myself. Can you explain this part?

[djphoenix]

...aaaand this is my miss. This is part of esptool.py with very dumb section start detection from symbol table, where _data_start is not exactly where .data section starts. I have merged .rodata and .data segments, so .rodata placed before and pads _data_start for 5e0 bytes. Very strange logic, so...

[djphoenix]

...So, without iROM checksum I successfully boot firmware that built without touch of esptool2. Now single miss is iROM checksum adjustment, that is very interesting... So, if I right, (xor-sum of bytes in irom section) ^ checksum from esptool.py = checksum that rboot expect, right?

[djphoenix]

...So, I was totally fix it out with:

$(BINODIR)/firmware.irom.bin: $(IMGODIR)/firmware.out
  @ mkdir -p $(dir $@)
  @ $(OBJCOPY) --dump-section .irom0.text=$@ $<

$(BINODIR)/firmware.tmp.bin: $(IMGODIR)/firmware.out
  @ mkdir -p $(dir $@)
  @ esptool.py elf2image -o $@ -e 2 $<

$(BINODIR)/firmware.bin: $(BINODIR)/firmware.irom.bin $(BINODIR)/firmware.tmp.bin
  @ cp $(BINODIR)/firmware.tmp.bin $@
  @ truncate -s -1 $@
  @ tail -c1 $(BINODIR)/firmware.tmp.bin | \
    cat $(BINODIR)/firmware.irom.bin - | \
    xxd -ps -c1 | \
    awk '{sum=xor(sum,strtonum("0x"$$1))} END {printf"%x",sum}' | \
    xxd -ps -r >> $@

[raburton]

This might be useful for some, but as esptool2 is a small platform independent c program that does the job much more simply and neatly (and was specifically written for use here and is well tested) I'll stick with that for official use. If people need to avoid it for some reason they can use your instructions so thanks for documenting them here.