search

v923z / micropython-ulab

a numpy-like fast vector module for micropython, circuitpython, and their derivatives
https://micropython-ulab.readthedocs.io/en/latest
MIT License
425 stars 116 forks source link

Switch to `cmake` on the ESP32 port? #372

Closed teuler closed 3 years ago

teuler commented 3 years ago

Is there a plan to enable including ulab into MicroPython >1.14 for the ESP? Thanks!

v923z commented 3 years ago

@teuler I am not against it, but I don't have the resources at the moment. Do you think you could update the README, and the build script?

teuler commented 3 years ago

Unfortunately, not sure if I know enough about cmake ...

v923z commented 3 years ago

Unfortunately, not sure if I know enough about cmake ...

The cmake fragment is already in ulab, the question is, whether the rest of the build script is correct. The raspberry pico is known to work with cmake: https://github.com/v923z/micropython-ulab/blob/master/build/rp2.sh

teuler commented 3 years ago

This is my take on that:

The following procedure builds the MicroPython firmware for ESP32 microcontrollers based on the latest version of MicroPython (currently 1.15) and includes ulab.

It has been tested with Ubuntu 20.04 installed in the Linux subsystem of Windows 10 (64 bit).

Note: The only limitation I noticed is that deploying the firmware seems not to work with the standard baud rate 460800, however, reducing this to 230400 works reliably (see below).

  1. The first part largely follows the instructions for the ESP32 in the MicroPython repository.

    export BUILD_DIR=$(pwd)

git clone https://github.com/v923z/micropython-ulab.git ulab
git clone https://github.com/micropython/micropython.git

cd $BUILD_DIR/micropython/

git clone -b v4.0.2 --recursive https://github.com/espressif/esp-idf.git

cd esp-idf
./install.sh    
. ./export.sh

cd $BUILD_DIR/micropython/
make -C mpy-cross
cd $BUILD_DIR/micropython/ports/esp32
make submodules

    Run make to check if the installation was successful.

  2. In the following steps, we generate a configuration file and a script to facilitate re-building the firmware and to include ulab:

    Still in $BUILD_DIR/micropython/ports/esp32 create a makefile e.g. using nano makefile:

    BOARD = GENERIC
PORT = /dev/ttyS4
BAUD = 460800
USER_C_MODULES = $(BUILD_DIR)/ulab/code/micropython.cmake

include Makefile

    BOARD selects the board type (e.g. GENERIC or TINYPICO etc.); PORT needs to be changed according to COM port your board connects to (here, e.g. COM4 under Windows). Now, MicroPython can be re-built with ulab and deployed:

    make clean
make
make deploy

    If deploy fails, try changing the baud rate in makefile to 230400. This seems to be an issue when deploying from the Windows 10 Linux subsystem.

    The export of the esp-idf paths is here not permanent. To facilitate re-building the firmware in a new session, the following script helps to set the stage. In $BUILD_DIR create a file e.g. using nano prep.sh containing:

    export BUILD_DIR=$(pwd)
cd $BUILD_DIR/micropython/esp-idf
source export.sh
cd $BUILD_DIR/micropython/ports/esp32

    When starting a new session, source prep.sh from your build folder (. prep.sh), which will export the esp-idf paths and bring you to directly to $BUILD_DIR/micropython/ports/esp32.

v923z commented 3 years ago

@teuler Many thanks, Thomas! Do you want to turn this into a PR, or should I just copy your comments into the README?

teuler commented 3 years ago

Why don't you just add it to the README. It would be good if someone else tested it.

v923z commented 3 years ago

Why don't you just add it to the README. It would be good if someone else tested it.

OK, will do. Thanks again!

teuler commented 3 years ago

RE: Ubuntu 18.04: Not as straight forward as I earlier thought: The build process requires CMake >= 3.12, but for Ubuntu 18.04 CMake 3.10.2 seems to be the highest version. Can probably be solved but I don't want to spend my time here as it works fine with Ubuntu 20.04.

v923z commented 3 years ago

RE: Ubuntu 18.04: Not as straight forward as I earlier thought: The build process requires CMake >= 3.12, but for Ubuntu 18.04 CMake 3.10.2 seems to be the highest version. Can probably be solved but I don't want to spend my time here as it works fine with Ubuntu 20.04.

I think this is OK, and this has nothing to do with ulab or micropython: if the OS is not equipped with the required tools, then it is not our responsibility.

teuler commented 3 years ago

Another note:

When selecting BOARD=TINYPICO, the firmware is built but deploy fails, because with ulab, the firmware is too large for the standard partition. To address this issue:

  1. Add a new partition table file named partitions_ulab.cvs to $BUILD_DIR/micropython/ports/esp32/. It increases the size of the partition factory by 128 kB and decreases the size vfs by the same amount.

    # Notes: the offset of the partition table itself is set in
# $ESPIDF/components/partition_table/Kconfig.projbuild and the
# offset of the factory/ota_0 partition is set in makeimg.py
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 0x200000,
vfs,      data, fat,     0x220000, 0x180000,

  2. Change sdkconfig.board in $BUILD_DIR/micropython/ports/esp32/boards/TINYPICO by adding following two lines:

    CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_ulab.csv"

  3. Finally, make sure that makefile in $BUILD_DIR/micropython/ports/esp32/ contains the correct board type (BOARD = TINYPICO).

v923z commented 3 years ago

@teuler Thomas, many thanks for your comments! I have incorporated them in https://github.com/v923z/micropython-ulab/pull/376. If you have any corrections, please, open a new issue! Closing now.