search

libretiny-eu / esphome-kickstart

Precompiled LibreTiny-ESPHome firmware allowing identification of device pins and OTA upgrading
35 stars 4 forks source link

Creating a custom flash uf2 file. #6

Closed protectivedad closed 10 months ago

protectivedad commented 10 months ago

Is there supposed to be an easy way to create a UF2 file with a custom app size. The boot log of my device looks like:

ROM:[V0.1]
FLASHRATE:4
BOOT TYPE:0 XTAL:40000000
IMG1 DATA[1128:10002000]
IMG1 ENTRY[800053f:100021ef]
IMG1 ENTER
CHIPID[000000f6]
read_mode idx:2, flash_speed idx:0
calibration_result:[1:3:13][b:d]
calibration_result:[2:13:7][1:d]
calibration_result:[3:7:5][1:7]
calibration_ok:[2:13:7]
FLASH CALIB[NEW OK]
OTA2 ADDR[80ba000]
OTAx SELE[ffffffff]
OTA1 USE
IMG2 DATA[0x807f2d0:3908:0x10005000]
IMG2 SIGN[RTKWin(10005008)]
IMG2 ENTRY[0x10005000:0x803a5d5]
System_Init1
System_Init2
I [      0.000] LibreTiny v1.4.0 on generic-rtl8710bn-2mb-788k, compiled at Oct 26 2023 08:11:53, GCC 10.3.1 (-Os)

Notice that the OTA2 ADDR is 0x80ba000 which gives an app size of 700KB, so I am trying to create a uf2 file using that app size.

I used the kickstart-rtl8710bn-2mb-788k.yaml to create:

substitutions:
  name: kickstart-rtl8710bn-2mb-788k
  import_url: github://libretiny-eu/esphome-kickstart/yaml/import-rtl8710bn-2mb-788k.yaml@master

packages:
  base: !include kickstart-base.yaml

esphome:
  platformio_options:
    custom_flash.app: 0xAF000

rtl87xx:
  board: generic-rtl8710bn-2mb-788k
#  loglevel: debug

select:
  - platform: template
    id: !extend pin
    options:
      - Select PIN
      - PA06 / FCS
      - PA07 / FD1
      - PA08 / FD2
      - PA09 / FD0
      - PA10 / FSCK
      - PA12 / PWM3
      - PA15 / PWM1 / SWDIO
      - PA18 / UART0_RX / I2C1_SCL / SPI0_SCK / SPI1_SCK
      - PA19 / UART0_CTS / I2C0_SDA / SPI0_CS / SPI1_CS
      - PA22 / UART0_RTS / I2C0_SCL / SPI0_MISO / SPI1_MISO / PWM5
      - PA23 / UART0_TX / I2C1_SDA / SPI0_MISO / SPI1_MISO / PWM0
      - PA29 / UART2_RX / I2C0_SCL / PWM4
      - PA30 / UART2_TX / I2C0_SDA / PWM4

output:
  - platform: gpio
    pin: PA14
    id: led_error
  - platform: gpio
    pin: PA05
    id: led_status

switch:
  - platform: gpio
    name: "Relay"
    pin: PA00
    restore_mode: RESTORE_DEFAULT_OFF
  - platform: output
    name: "Green LED"
    output: led_status
    restore_mode: ALWAYS_ON
  - platform: output
    name: "Red LED"
    output: led_error
    restore_mode: DISABLED

binary_sensor:
  - platform: gpio
    pin:
      number: PA22
      inverted: true
    name: "Input"

But it errors with:

CUSTOM OPTIONS:
 - flash:
     - app = 716800
 - fw_name = esphome
 - fw_version = 2023.10.0-dev
CUSTOM FLASH LAYOUT:
ValueError: invalid literal for int() with base 16: '':
  File "/home/anthony/.local/lib/python3.10/site-packages/platformio/builder/main.py", line 173:                                                             
    env.SConscript("$BUILD_SCRIPT")                                                                                                                          
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:                                               
    return _SConscript(self.fs, *files, **subst_kw)                                                                                                          
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:                                               
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)                                                                                    
  File "/home/anthony/.platformio/platforms/libretiny/builder/main.py", line 89:                                                                             
    target_elf = env.BuildProgram()                                                                                                                          
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:                                                       
    return self.method(*nargs, **kwargs)                                                                                                                     
  File "/home/anthony/.local/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 61:                                                    
    env.ProcessProgramDeps()                                                                                                                                 
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:                                                       
    return self.method(*nargs, **kwargs)                                                                                                                     
  File "/home/anthony/.local/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 121:                                                   
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))                                                                                                             
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:                                                       
    return self.method(*nargs, **kwargs)                                                                                                                     
  File "/home/anthony/.local/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 342:                                                   
    SConscript(env.GetFrameworkScript(name), exports="env")                                                                                                  
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:                                               
    return method(*args, **kw)                                                                                                                               
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:                                               
    return _SConscript(self.fs, *files, **subst_kw)                                                                                                          
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:                                               
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)                                                                                    
  File "/home/anthony/.platformio/platforms/libretiny/builder/frameworks/arduino.py", line 15:                                                               
    env.SConscript("base.py")                                                                                                                                
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:                                               
    return _SConscript(self.fs, *files, **subst_kw)                                                                                                          
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:                                               
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)                                                                                    
  File "/home/anthony/.platformio/platforms/libretiny/builder/frameworks/base.py", line 28:                                                                  
    env.PrintInfo(platform, board)                                                                                                                           
  File "/home/anthony/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:                                                       
    return self.method(*nargs, **kwargs)                                                                                                                     
  File "/home/anthony/.platformio/platforms/libretiny/builder/utils/env.py", line 130:                                                                       
    length = int(length, 16)

I tried using the libretiny-esphome but it gives the same error. Any help resolving this is appreciated.

kuba2k2 commented 10 months ago

You're on the right path here. Thumbs up for reading the documentation!

However, I think the partition layout needs the start address AND length (separated by a colon). So, it'd be "0xB000:0xAF000". The partition is named "ota1" (or "xip1" I'm not sure), not "app". Make sure to also add the appropriate record for OTA2 partition - start offset 0xBA000, same length.

protectivedad commented 10 months ago

Thanks for taking the time. Using:

  platformio_options:
    custom_flash.ota1: "0x0B000:0xAF00"
    custom_flash.ota2: "0xBA000:0xAF00"

Gives the error:

HARDWARE: RTL8710BN 125MHz, 256KB RAM, 788KB Flash
 - framework-arduino-api @ 2022.8.24+sha.237b10a 
 - framework-realtek-amb1 @ 0.0.0+v2022.06.21.sha.c4e44ef 
 - library-flashdb @ 1.2.0+sha.d5c892f 
 - library-freertos @ 8.1.2+sha.776ae6c 
 - library-freertos-port @ 2023.5.23+sha.a917d93 
 - library-lwip @ 2.1.3-amb1+sha.6297b80 
 - library-printf @ 6.1.0+sha.28a79bd 
 - library-uf2ota @ 5.0.0+sha.f955412 
ValueError: invalid literal for int() with base 0: '0x0B000:0xAF000':
  File "/home/anthony/.local/lib/python3.10/site-packages/platformio/builder/main.py", line 173:                                                             
    env.SConscript("$BUILD_SCRIPT")                                                                                                                          
...

Tried with xip1 and xip2 same error. Comma, semicolon, all the same.

I looked trough the library code I see the FLASH_OTA[1,2]_[OFFSET,LENGTH] variables but where they are assigned I can't find. It's like magic.

kuba2k2 commented 10 months ago

I remember now. It's "+", not a colon.

protectivedad commented 10 months ago

Yeah, just figured that also. Still doesn't work. The python code is slightly broken. The flash.py (library-uf2ota) can only handle an offset, but the env.py is broken if FLASH_IS_CUSTOM is set.

Just adding:

    custom_flash.ota2: "0x0BA000"

will create a usable `board.json:

        "ota1": "0x00B000+0xAF000",
        "ota2": "0x0BA000+0xDB000",

I'm fine with a larger second ota2 partition. But it flash.py sets FLASH_IS_CUSTOM which breaks env.py.

kuba2k2 commented 10 months ago

Right. It does auto-calculation of other partitions; I forgot that the value in your yaml should be just the offset, without length (I've never actually used that option).

And yes, it seems that env.py is broken - it tries to .partition() the v variable, which is probably a copy-paste error or something.

Until I fix this, you can use board_flash.app and set it to both offset and length values, with + in between. You have to set both partitions then. It won't recalculate the entire partition layout, so it might cause some unused space, or even overlapping partitions.

kuba2k2 commented 10 months ago

I see you beat me to it, haha.. merged the PR, thanks!

protectivedad commented 10 months ago

No problem.