WebUI - Python throw "468@code.py MemoryE" when uploading big macros settings

[x-magic]

When there are too many macros loaded on the device, and try to upload similar or larger payload worth of macros to the device through WebUI, CircuitPython will crash with the error 468@code.py MemoryE (unfortunately I haven't figured out a way to see the rest of the error message and this is all displayed on the OLED).

Steps to reproduce: Macros in question: macros (7).json

1. Load the large macros into WebUI, connect macropad
2. Load the macros into macropad
3. Store macros on macropad
4. Load the macros again
5. Macropad will crash with error 468@code.py MemoryE

When an empty macro config is uploaded to the macropad, stored, then load the large macros again, macros would upload to macropad as expected. Also when USB storage is enabled, large macros would upload to macropad as expected.

[mchilli]

Hey, I almost thought that this problem would occur at some point. It's that “json.load” holds the complete string in memory to create the object. And that leads to the “MemoryError”. But at the moment I can't think of a nice solution to get around this problem. Unfortunately, microcontrollers do not have much memory available. Maybe it would be necessary to process the string serially to solve the memory problem?

[x-magic]

Maybe store each page as its own object/file and load/write one object at a time? Divide and conquer 😎

Or instead, store bitstream of macros.json into storage then load from there instead of holding it in memory, instead of having dedicated upload and store? Like offloading memory-intensive operations to WebUI instead of the microcontroller.

[mchilli]

Hey, would you like to take a look at the new update 1.4 and give me some feedback? In my tests it can handle about 400 key configurations, depending on the size of the macros. I think that's enough for such a small device 😉 . You must also use the updated WebUI to transfer the macros, but you can use the old save files.