Fix flash_params in images during download?

[rogercollins]

First, thank you for this tool! I ported the download piece to a bare metal MCU and could update production hardware in the field. But that quit working when I got modules with flash larger than 4 mbit (the default). That's because the flash params are not set in the boot image.

I'll share the result of a hard-coded fix to get feedback.

The boot files need the flash parameters in their headers to be set before downloading. Is the strategy of esptool-ck for user to convert those binaries into other binary with parameters set and then download using the latter? If so, how do you do that conversion?

I can install the 1.5.3 SDK binaries on a 16 mbit (2 MB) flash with esptool.py like so:

esptool.py -p /dev/tty.usbserial-FTHC2V0I write_flash -fs 16m-c1 0x0 boot_v1.5.bin 0x1000 user1.2048.new.5.bin 0x1fc000 esp_init_data_default.bin 0xfe000 blank.bin 0x1fe000 blank.bin

Here is the parallel using esptool-ck:

esptool -bz 16M -cp /dev/tty.usbserial-FTHC2V0I -ca 0x0 -cf boot_v1.5.bin -ca 0x1000 -cf user1.2048.new.5.bin -ca 0x1fc000 -cf esp_init_data_default.bin -ca 0xfe000 -cf blank.bin -ca 0x1fe000 -cf blank.bin

I got it to work with esptool-ck by making this change:

diff --git a/espcomm/espcomm.c b/espcomm/espcomm.c
index 52e0474..f84565a 100644
--- a/espcomm/espcomm.c
+++ b/espcomm/espcomm.c
@@ -415,6 +415,25 @@ bool espcomm_upload_mem(uint8_t* src, size_t size, const char* source_name)
         size -= write_size;
         src += write_size;

+        /*
+         * From esptool.py
+         * # Fix sflash config data.
+         * if address == 0 and image[0] == '\xe9':
+         *   print 'Flash params set to 0x%02x%02x' % (flash_mode, flash_size_freq)
+         *   image = image[0:2] + flash_params + image[4:]
+         */
+        if (espcomm_address == 0 && count == 0 && (flash_packet[4] & 0xff) == 0xe9)
+        {
+            /* replace flash_params in header of image
+             * from esptool.py...
+             * flash_mode = {'qio':0, 'qout':1, 'dio':2, 'dout': 3}[args.flash_mode]
+             * flash_size_freq = {'4m':0x00, '2m':0x10, '8m':0x20, '16m':0x30, '32m':0x40, '16m-c1': 0x50, '32m-c1':0x60, '32m-c2':0x70}[args.flash_size]
+             * flash_params = struct.pack('BB', flash_mode, flash_size_freq)
+             */
+            LOGINFO("setting flash_parms in boot to qio, 16m-c1");
+            flash_packet[4] = (flash_packet[4] & 0x0000ffff) | 0x50000000;
+        }
+
         send_packet.checksum = espcomm_calc_checksum((unsigned char *) (flash_packet + 4), BLOCKSIZE_FLASH);
         res = espcomm_send_command(FLASH_DOWNLOAD_DATA, (unsigned char*) flash_packet, BLOCKSIZE_FLASH + 16, 0);

What do you think? Does this tool need a generic way of doing the above?

[rogercollins]

A related question. Can someone explain the difference between 16m and 16m-c1? What does the -c1 mean? esptool.py supports 16m-c1 and espressif documentation uses it, but I could not find a description of it.

[igrr]

This is something that esptool.py and esptool-ck do differently. Both can generate binary images, and both can upload them. But flash size setting is applied at different stages:

• esptool.py doesn't set binary image header when generating the image, but it changes the header when the image is uploaded.
• esptool-ck, on the other hand, sets correct value in binary image header when the image is generated. No change is performed to the image as it is uploaded.

My assumption is that upload process should not modify anything behind the scenes, because that will certainly lead to some "wtf?" moments.

If you need to use esptool-ck to upload images generated with esptool.py, you may patch the flash size/mode byte from the command line.

Check the size in original file (4th byte):

$ xxd -g1 -l4  sketch_jun08a.ino.bin
0000000: e9 01 02 00

Change the size:

$ printf '\x04' | dd of=sketch_jun08a.ino.bin bs=1 seek=3 count=1 conv=notrunc
1+0 records in
1+0 records out
1 bytes transferred in 0.000017 secs (59075 bytes/sec)

Check the result:

$ xxd -g1 -l4  sketch_jun08a.ino.bin
0000000: e9 01 02 04

Regarding 16m vs 16m-c1, i have traced this to https://github.com/themadinventor/esptool/commit/a1ea72daaa47b33023108049bfd1820046e017f1, but not sure exactly what these modes are.

[rogercollins]

Sorry for the delay! I accept the different styles for when and how to set the flash size/mode byte. I prefer to have an option to set these values inline rather than manage another binary file type in my build, but I admit the difference is trivial. Thank you for the explanation.