Create new TinyUSB NO-OTA Partition with decreased FATFS size

[brentru]

Board

ESP32-S2

Device Description

Adafruit FunHouse

Hardware Configuration

n/a

Version

latest master (checkout manually)

IDE Name

Arduino IDE

Operating System

macOS

Flash frequency

default

PSRAM enabled

yes

Upload speed

default

Description

@hathach When building WipperSnapper with LVGL on the FunHouse, we are running out of flash space again:

Sketch uses 1441826 bytes (100%) of program storage space. Maximum is 1441792 bytes.
  Global variables use 74772 bytes (22%) of dynamic memory, leaving 252908 bytes for local variables. Maximum is 327680 bytes.

This is attributed to the 1.3MB FatFS partition on the "TinyUF2 NOOTA" partition.

Could we either create a new partition reduce the FatFS partition to 256KB (giving WipperSnapper enough space for JSON files and any graphical/sound file should we need one), or modify the existing one?

This is what I'd like the new partition to look like:

App Partition: 3840KB
FATFS Partition: 256KB

cc @ladyada

Sketch

https://github.com/adafruit/Adafruit_Wippersnapper_Arduino/blob/main/examples/Wippersnapper_demo/Wippersnapper_demo.ino

Debug Message

see GH:A log

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• [X] I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[brentru]

@hathach If you have any questions (I know you're coming back from a holiday), please let me know

[hathach]

currently tinyuf2 is at 0x2d0000, to increase applicationn partition, we will also need to change the offset of tinyuf2 as well. Note: that would cause some hiccup when switching between arduino <-> circuitpython (not sure)

current partition

nvs,      data, nvs,      0x9000,  20K,
otadata,  data, ota,      0xe000,  8K,
ota_0,    app,  ota_0,   0x10000,  2816K,
uf2,      app,  factory,0x2d0000,  256K,
ffat,     data, fat,    0x310000,  960K,

new partitionn with fatfs shrinked to 256KB, does it look ok to you ?

nvs,      data, nvs,      0x9000,  20K,
otadata,  data, ota,      0xe000,  8K,
ota_0,    app,  ota_0,   0x10000,  3520K,
uf2,      app,  factory,0x380000,  256K,
ffat,     data, fat,    0x3c0000,  256K,

[hathach]

another question: do we need to add this partition scheme for all other S2,S3 with 4MB flash, or we only need to do this with funhouse board only ?

[brentru]

new partitionn with fatfs shrinked to 256KB, does it look ok to you ?

@hathach This looks OK

Note: that would cause some hiccup when switching between arduino <-> circuitpython (not sure)

Currently when switching from WipperSnapper/Arduino to CircuitPython, one needs to put the board back in bootloader mode and load the circuitpython UF2. What would change within this process if the offset is moved to 0x380000?

another question: do we need to add this partition scheme for all other S2,S3 with 4MB flash, or we only need to do this with funhouse board only ?

All S2/S3 boards with 4MB flash, we'll be putting LVGL on these boards running WipperSnapper

[hathach]

Currently when switching from WipperSnapper/Arduino to CircuitPython, one needs to put the board back in bootloader mode and load the circuitpython UF2. What would change within this process if the offset is moved to 0x380000?

That won't be enough unfortunately, since tinyuf2 will only make changes to the ota contents, not the parition. Circuitpython will run, but with the exact same layout i.e 3.5M app and 256KB of fatfs. To get back to 1MB app + 960KB fatfs, user need to select the partiion scheme from arduino and flash a sketch (e.g blinky) then load circuitpython.

All S2/S3 boards with 4MB flash, we'll be putting LVGL on these boards running WipperSnapper

Got it, I will proceed with the above parition scheme and make PR to arduino-esp32 upstream.

[brentru]

That won't be enough unfortunately, since tinyuf2 will only make changes to the ota contents, not the parition. Circuitpython will run, but with the exact same layout i.e 3.5M app and 256KB of fatfs. To get back to 1MB app + 960KB fatfs, user need to select the partiion scheme from arduino and flash a sketch (e.g blinky) then load circuitpython.

@ladyada is this OK? I think it may be too difficult for a beginner user to get back from Arduino/WS to CircuitPython on this proposed partition scheme

[hathach]

maybe we could document it somewhere in the WipperSnapper when using with lvgl. It isn't complicated, it may just a bit confusinng if user does not pay attention to.

[ladyada]

id say start by making a new board definition in wipper specifically for a 'lvgl' build. we can have it for advanced users who are OK with uploading a bin to restore circuitpy/arduino access

[hathach]

@brentru @ladyada I have made an push to add a new partition scheme with 3.5MB applicaiton (tinyuf2 offset is changed to 0x380000 to make room for the app), please give a try https://github.com/adafruit/arduino-esp32/tree/add-tinyuf2-partiion-4mb-ota3mb . If that works well I will submit this as PR to official epsressif repo

PS: I also take the chance to update tinyuf2 to latest version 0.18.1 since I found out funhouse and other board still use very old release (0.10.0 or so).

PSS: as mentioned above: partition scheme/table can only be changed by esptool.py with either arduino IDE or using command line. TinyUF2 drag and drop can only change the application partition contetns (not the partition table). Though since switching it is rather easy, maybe we put a note in the wipper lvgl guide.

[hathach]

@brentru let me know if this PR works as expected with your testing

[brentru]

@hathach I'll test this today, last week was very busy

[brentru]

id say start by making a new board definition in wipper specifically for a 'lvgl' build. we can have it for advanced users who are OK with uploading a bin to restore circuitpy/arduino access

@ladyada I can and will take this route, it'll work like this:

1) I will add documentation to Learn on how to upload a bin to restore circuitpy/arduino access for this build 1.1) We can add a future step where there's a path for restoring the bootloader directly from io.adafruit.com, we have access to webserial and firmware images there. This would be a task for Loren. 2) For initially getting this build onto a board (new or existing), we can't use UF2. We'll want this new definition to use the web uploader path via io.adafruit.com's New Boards feature.

[brentru]

@hathach Thanks for your work on this, I'm outlining what I need from our end too. I think it's ready to PR into the Espressif repo.

For a usage comparison, we're at 101% of program storage on regular NOOTA vs 39% storage on this partition scheme!

Sketch uses 1429574 bytes (39%) of program storage space. Maximum is 3604480 bytes.
Global variables use 74988 bytes (22%) of dynamic memory, leaving 252692 bytes for local variables. Maximum is 327680 bytes.

[hathach]

uhm, @brentru the regular noota has 2816K (2.8M) and it should be able to work since your sketch only occupies 1.4M.

[brentru]

@hathach Oops - thank you for bringing that to my attention. I think the CI runner for WipperSnapper is targeting FunHouse but it isn’t using the NOOTA partition. I thought it was but I did not verify closely enough.

You are right, it fits on regular NOOTA

Sketch uses 1434418 bytes (49%) of program storage space. Maximum is 2883584 bytes.
Global variables use 74988 bytes (22%) of dynamic memory, leaving 252692 bytes for local variables. Maximum is 327680 bytes.

I am going to have Tyeth update the WipperSnapper CI to target FunHouse and NOOTA partition.

[hathach]

@brentru so we don't really need to extend the firmware region to 3MB as this PR since 2MB (noota) is large enough right ?

[brentru]

@hathach Correct, we could close out this PR