ESP32 Core 1 Threading

[dcmcshan]

I would like to run a thread on core 1 of ESP32S3. Here is an example for us to work with. Ideally, I'd like to be able to run the thread on core1 at 1000Hz. It will collect some high rate data, and core0 will display it.

How do I use your cleverness to get thread_core1 actually on core1?

import _thread
import time

# Shared counter variable
counter = 0

def thread_core1():
    print("Starting thread_core1")
    global counter
    while True:
        counter += 1
        time.sleep_ms(10)

def thread_core0():
    print("Starting thread_core0")
    while True:
        print(f"Counter value: {counter}")
        time.sleep(1)

# Start thread on Core 1
_thread.start_new_thread(thread_core1, ())

# Start thread on Core 0 (main core)
_thread.start_new_thread(thread_core0, ())

[kdschlosser]

Well..... already built into the binding is the ability to turn on multi core threading. However you will not get to decide what gets run on what core. All threads run on core 1 and the main loop runs on core 0. It has not been tested and the GIL gets disabled in order for it to work. There is more development I need to do to expose being able to send notifications and adding different lock types and waits that don't have a spinning wheels. Things like that. I will mess about with it this evening to see what I can do with it.

[dcmcshan]

How do I turn on multicore threading?

[kdschlosser]

It's not 100% complete yet. You are more then welcome to test it to see if it even works at all.

You have to be really careful with having multiple threads writing to the same object. There is no GIL protection at all so you are going to need to make use of locks to synchronize the access.

add --dual-core-threads to your build command.

[dcmcshan]

My plan is to have the data flow one way. I'll give it a try today!

[dcmcshan]

Doesn't build. Doesn't error either? Can you review my command line and the output?
Thank you!

% python3 make.py esp32 clean submodules BOARD=ESP32_GENERIC_S3 BOARD_VARIANT=SPIRAM_OCT --flash-size=16 DISPLAY=ST7735 --usb-otg --dual-core-threads > build.txt
makeversionhdr.py: Warning: No git repo or tag info available, falling back to mpconfig.h version info.

build.txt

I'll try and check out and rebuild from scratch rather than just updating

[kdschlosser]

It will build. For whatever reason you are having an issue with your connection to github. Just wait and it will eventually go. Sometimes that happens.

[dcmcshan]

Did the build command change? This was the build command I've been using forever, and it doesn't seem to actually build...

I have cloned from main.

python3 make.py esp32 clean submodules BOARD=ESP32_GENERIC_S3 BOARD_VARIANT=SPIRAM_OCT --flash-size=16 DISPLAY=ST7735 --usb-otg build.txt

[kdschlosser]

as I said. You have to wait for it to finish. It will it's going to take some time due to a routing issue to github..

[dcmcshan]

No, it's definitely "done". Perhaps it failed silently?

Compiling....
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
Updating submodules:

danielmcshan@Daniels-Laptop lvgl_micropython % 

[dcmcshan]

Works fine without submodules target...

python3 make.py esp32 BOARD=ESP32_GENERIC_S3 BOARD_VARIANT=SPIRAM_OCT V=1 --flash-size=16 DISPLAY=ST7735 --usb-otg --dual-core-threads

So that works well enough.

I am now running my test program. It works well enough, but is there a way I can confirm that it's actually running on Core 1? If all threads were running on Core 1, and REPL on Core 0, I would expect REPL to be responsive, and it isn't...

import _thread
import time

# Shared counter variable
counter = 0

def thread_core1():
    print("Starting thread_core1")
    global counter
    while True:
        counter += 1
        time.sleep_ms(1)

def thread_core0():
    print("Starting thread_core0")
    while True:
        print(f"Counter value: {counter}")
        time.sleep(1)

# Start thread on Core 1
_thread.start_new_thread(thread_core1, ())

# Start thread on Core 0 (main core)
_thread.start_new_thread(thread_core0, ())

[dcmcshan]

Actually... this does seem to work, though I cannot be sure that it's using Core 1.

import _thread
import time

# Shared counter variable
counter = 1

def thread_core1():
    print("Starting thread_core1")
    global counter
    while True:
        counter += 1
        time.sleep_us(10)

t0 = time.ticks_ms()
def thread_core0():
    global t0
    print("Starting thread_core0")
    start_time = time.ticks_ms()
    while True:
        current_time = time.ticks_ms()
        dt = time.ticks_diff(current_time, t0)
        t0 = current_time
        elapsed_time = time.ticks_diff(current_time, start_time) / 1000  # in seconds
        frequency = counter / elapsed_time if elapsed_time > 0 else 0  # Avoid division by zero
        print(f"Counter value: {counter:8} {dt:8}ms {frequency:8.2f}Hz")
        time.sleep(1)

# Start thread on Core 1
_thread.start_new_thread(thread_core1, ())

# Start thread on Core 0 (main core)
_thread.start_new_thread(thread_core0, ())

Counter value:  9294163     1001ms 101747.86Hz

>>> Counter value:  9394909     1006ms 101730.45Hz
Counter value:  9497530     1010ms 101729.09Hz
Counter value:  9599405     1002ms 101728.45Hz
Counter value:  9701396     1001ms 101730.13Hz
Counter value:  9803377     1002ms 101730.63Hz
Counter value:  9905679     1005ms 101731.30Hz

[dcmcshan]

I tried adding this to esp32/modules and defining MODULE_CORE_INFO_ENABLED in mpconfgport.h, but I think your build process must overwrite that file, as it's not defined after I build?

#include "py/obj.h"
#include "py/runtime.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

// Function to get the core ID of the current running task
STATIC mp_obj_t get_core_id(void) {
    int core_id = xPortGetCoreID();  // Get the current core ID
    return mp_obj_new_int(core_id);  // Return the core ID as an integer
}
STATIC MP_DEFINE_CONST_FUN_OBJ_0(get_core_id_obj, get_core_id);

// Module definition
STATIC const mp_rom_map_elem_t core_module_globals_table[] = {
    { MP_ROM_QSTR(MP_QSTR_get_core_id), MP_ROM_PTR(&get_core_id_obj) },
};
STATIC MP_DEFINE_CONST_DICT(core_module_globals, core_module_globals_table);

const mp_obj_module_t core_module = {
    .base = { &mp_type_module },
    .globals = (mp_obj_dict_t *)&core_module_globals,
};

// Register the module to make it available in Python
MP_REGISTER_MODULE(MP_QSTR_core_info, core_module, MODULE_CORE_INFO_ENABLED);

[kdschlosser]

you don't add that kind of a feature to a header file because the header files do not get compiled. only the source files do. You would be better to add it to modesp32.c

#include <stdio.h>
#include <string.h>

#include <time.h>
#include <sys/time.h>
#include "soc/rtc_cntl_reg.h"
#include "driver/gpio.h"
#include "driver/adc.h"
#include "esp_heap_caps.h"
#include "multi_heap.h"

#include "py/nlr.h"
#include "py/obj.h"
#include "py/runtime.h"
#include "py/mphal.h"
#include "shared/timeutils/timeutils.h"
#include "modmachine.h"
#include "machine_rtc.h"
#include "modesp32.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

// These private includes are needed for idf_heap_info.
#define MULTI_HEAP_FREERTOS
#include "../multi_heap_platform.h"
#include "../heap_private.h"

static mp_obj_t esp32_wake_on_touch(const mp_obj_t wake) {

    if (machine_rtc_config.ext0_pin != -1) {
        mp_raise_ValueError(MP_ERROR_TEXT("no resources"));
    }
    // mp_raise_msg(&mp_type_RuntimeError, MP_ERROR_TEXT("touchpad wakeup not available for this version of ESP-IDF"));

    machine_rtc_config.wake_on_touch = mp_obj_is_true(wake);
    return mp_const_none;
}
static MP_DEFINE_CONST_FUN_OBJ_1(esp32_wake_on_touch_obj, esp32_wake_on_touch);

static mp_obj_t esp32_wake_on_ext0(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {

    if (machine_rtc_config.wake_on_touch) {
        mp_raise_ValueError(MP_ERROR_TEXT("no resources"));
    }
    enum {ARG_pin, ARG_level};
    const mp_arg_t allowed_args[] = {
        { MP_QSTR_pin,  MP_ARG_OBJ, {.u_obj = mp_obj_new_int(machine_rtc_config.ext0_pin)} },
        { MP_QSTR_level,  MP_ARG_BOOL, {.u_bool = machine_rtc_config.ext0_level} },
    };
    mp_arg_val_t args[MP_ARRAY_SIZE(allowed_args)];
    mp_arg_parse_all(n_args, pos_args, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, args);

    if (args[ARG_pin].u_obj == mp_const_none) {
        machine_rtc_config.ext0_pin = -1; // "None"
    } else {
        gpio_num_t pin_id = machine_pin_get_id(args[ARG_pin].u_obj);
        if (pin_id != machine_rtc_config.ext0_pin) {
            if (!RTC_IS_VALID_EXT_PIN(pin_id)) {
                mp_raise_ValueError(MP_ERROR_TEXT("invalid pin"));
            }
            machine_rtc_config.ext0_pin = pin_id;
        }
    }

    machine_rtc_config.ext0_level = args[ARG_level].u_bool;
    machine_rtc_config.ext0_wake_types = MACHINE_WAKE_SLEEP | MACHINE_WAKE_DEEPSLEEP;

    return mp_const_none;
}
static MP_DEFINE_CONST_FUN_OBJ_KW(esp32_wake_on_ext0_obj, 0, esp32_wake_on_ext0);

static mp_obj_t esp32_wake_on_ext1(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {
    enum {ARG_pins, ARG_level};
    const mp_arg_t allowed_args[] = {
        { MP_QSTR_pins, MP_ARG_OBJ, {.u_obj = mp_const_none} },
        { MP_QSTR_level, MP_ARG_BOOL, {.u_bool = machine_rtc_config.ext1_level} },
    };
    mp_arg_val_t args[MP_ARRAY_SIZE(allowed_args)];
    mp_arg_parse_all(n_args, pos_args, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, args);
    uint64_t ext1_pins = machine_rtc_config.ext1_pins;

    // Check that all pins are allowed
    if (args[ARG_pins].u_obj != mp_const_none) {
        size_t len = 0;
        mp_obj_t *elem;
        mp_obj_get_array(args[ARG_pins].u_obj, &len, &elem);
        ext1_pins = 0;

        for (int i = 0; i < len; i++) {

            gpio_num_t pin_id = machine_pin_get_id(elem[i]);
            if (!RTC_IS_VALID_EXT_PIN(pin_id)) {
                mp_raise_ValueError(MP_ERROR_TEXT("invalid pin"));
                break;
            }
            ext1_pins |= (1ll << pin_id);
        }
    }

    machine_rtc_config.ext1_level = args[ARG_level].u_bool;
    machine_rtc_config.ext1_pins = ext1_pins;

    return mp_const_none;
}
static MP_DEFINE_CONST_FUN_OBJ_KW(esp32_wake_on_ext1_obj, 0, esp32_wake_on_ext1);

static mp_obj_t esp32_wake_on_ulp(const mp_obj_t wake) {
    if (machine_rtc_config.ext0_pin != -1) {
        mp_raise_ValueError(MP_ERROR_TEXT("no resources"));
    }
    machine_rtc_config.wake_on_ulp = mp_obj_is_true(wake);
    return mp_const_none;
}
static MP_DEFINE_CONST_FUN_OBJ_1(esp32_wake_on_ulp_obj, esp32_wake_on_ulp);

static mp_obj_t esp32_gpio_deep_sleep_hold(const mp_obj_t enable) {
    if (mp_obj_is_true(enable)) {
        gpio_deep_sleep_hold_en();
    } else {
        gpio_deep_sleep_hold_dis();
    }
    return mp_const_none;
}
static MP_DEFINE_CONST_FUN_OBJ_1(esp32_gpio_deep_sleep_hold_obj, esp32_gpio_deep_sleep_hold);

#if CONFIG_IDF_TARGET_ESP32

#include "soc/sens_reg.h"

static mp_obj_t esp32_raw_temperature(void) {
    SET_PERI_REG_BITS(SENS_SAR_MEAS_WAIT2_REG, SENS_FORCE_XPD_SAR, 3, SENS_FORCE_XPD_SAR_S);
    SET_PERI_REG_BITS(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_CLK_DIV, 10, SENS_TSENS_CLK_DIV_S);
    CLEAR_PERI_REG_MASK(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_POWER_UP);
    CLEAR_PERI_REG_MASK(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_DUMP_OUT);
    SET_PERI_REG_MASK(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_POWER_UP_FORCE);
    SET_PERI_REG_MASK(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_POWER_UP);
    esp_rom_delay_us(100);
    SET_PERI_REG_MASK(SENS_SAR_TSENS_CTRL_REG, SENS_TSENS_DUMP_OUT);
    esp_rom_delay_us(5);
    int res = GET_PERI_REG_BITS2(SENS_SAR_SLAVE_ADDR3_REG, SENS_TSENS_OUT, SENS_TSENS_OUT_S);

    return mp_obj_new_int(res);
}
static MP_DEFINE_CONST_FUN_OBJ_0(esp32_raw_temperature_obj, esp32_raw_temperature);

#else

// IDF 5 exposes new internal temperature interface, and the ESP32C3/S2/S3
// now have calibrated temperature settings in 5 discrete ranges.
#include "driver/temperature_sensor.h"

static mp_obj_t esp32_mcu_temperature(void) {
    static temperature_sensor_handle_t temp_sensor = NULL;
    float tvalue;
    if (temp_sensor == NULL) {
        temperature_sensor_config_t temp_sensor_config = TEMPERATURE_SENSOR_CONFIG_DEFAULT(-10, 80);
        ESP_ERROR_CHECK(temperature_sensor_install(&temp_sensor_config, &temp_sensor));
    }
    ESP_ERROR_CHECK(temperature_sensor_enable(temp_sensor));
    ESP_ERROR_CHECK(temperature_sensor_get_celsius(temp_sensor, &tvalue));
    ESP_ERROR_CHECK(temperature_sensor_disable(temp_sensor));
    return mp_obj_new_int((int)(tvalue + 0.5));
}
static MP_DEFINE_CONST_FUN_OBJ_0(esp32_mcu_temperature_obj, esp32_mcu_temperature);

#endif

static mp_obj_t esp32_idf_heap_info(const mp_obj_t cap_in) {
    mp_int_t cap = mp_obj_get_int(cap_in);
    multi_heap_info_t info;
    heap_t *heap;
    mp_obj_t heap_list = mp_obj_new_list(0, 0);
    SLIST_FOREACH(heap, &registered_heaps, next) {
        if (heap_caps_match(heap, cap)) {
            multi_heap_get_info(heap->heap, &info);
            mp_obj_t data[] = {
                MP_OBJ_NEW_SMALL_INT(heap->end - heap->start), // total heap size
                MP_OBJ_NEW_SMALL_INT(info.total_free_bytes),   // total free bytes
                MP_OBJ_NEW_SMALL_INT(info.largest_free_block), // largest free contiguous
                MP_OBJ_NEW_SMALL_INT(info.minimum_free_bytes), // minimum free seen
            };
            mp_obj_t this_heap = mp_obj_new_tuple(4, data);
            mp_obj_list_append(heap_list, this_heap);
        }
    }
    return heap_list;
}
static MP_DEFINE_CONST_FUN_OBJ_1(esp32_idf_heap_info_obj, esp32_idf_heap_info);

// Function to get the core ID of the current running task
STATIC mp_obj_t get_core_id(void)
{
    int core_id = xPortGetCoreID();  // Get the current core ID
    return mp_obj_new_int(core_id);  // Return the core ID as an integer
}

STATIC MP_DEFINE_CONST_FUN_OBJ_0(get_core_id_obj, get_core_id);

static const mp_rom_map_elem_t esp32_module_globals_table[] = {
    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_esp32) },

    // Function to get the core ID of the current running task
    { MP_ROM_QSTR(MP_QSTR_get_core_id), MP_ROM_PTR(&get_core_id_obj) },

    { MP_ROM_QSTR(MP_QSTR_wake_on_touch), MP_ROM_PTR(&esp32_wake_on_touch_obj) },
    { MP_ROM_QSTR(MP_QSTR_wake_on_ext0), MP_ROM_PTR(&esp32_wake_on_ext0_obj) },
    { MP_ROM_QSTR(MP_QSTR_wake_on_ext1), MP_ROM_PTR(&esp32_wake_on_ext1_obj) },
    { MP_ROM_QSTR(MP_QSTR_wake_on_ulp), MP_ROM_PTR(&esp32_wake_on_ulp_obj) },
    { MP_ROM_QSTR(MP_QSTR_gpio_deep_sleep_hold), MP_ROM_PTR(&esp32_gpio_deep_sleep_hold_obj) },
    #if CONFIG_IDF_TARGET_ESP32
    { MP_ROM_QSTR(MP_QSTR_raw_temperature), MP_ROM_PTR(&esp32_raw_temperature_obj) },
    #else
    { MP_ROM_QSTR(MP_QSTR_mcu_temperature), MP_ROM_PTR(&esp32_mcu_temperature_obj) },
    #endif
    { MP_ROM_QSTR(MP_QSTR_idf_heap_info), MP_ROM_PTR(&esp32_idf_heap_info_obj) },

    { MP_ROM_QSTR(MP_QSTR_NVS), MP_ROM_PTR(&esp32_nvs_type) },
    { MP_ROM_QSTR(MP_QSTR_Partition), MP_ROM_PTR(&esp32_partition_type) },
    { MP_ROM_QSTR(MP_QSTR_RMT), MP_ROM_PTR(&esp32_rmt_type) },
    #if CONFIG_IDF_TARGET_ESP32 || CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3
    { MP_ROM_QSTR(MP_QSTR_ULP), MP_ROM_PTR(&esp32_ulp_type) },
    #endif

    { MP_ROM_QSTR(MP_QSTR_WAKEUP_ALL_LOW), MP_ROM_FALSE },
    { MP_ROM_QSTR(MP_QSTR_WAKEUP_ANY_HIGH), MP_ROM_TRUE },

    { MP_ROM_QSTR(MP_QSTR_HEAP_DATA), MP_ROM_INT(MALLOC_CAP_8BIT) },
    { MP_ROM_QSTR(MP_QSTR_HEAP_EXEC), MP_ROM_INT(MALLOC_CAP_EXEC) },
};

static MP_DEFINE_CONST_DICT(esp32_module_globals, esp32_module_globals_table);

const mp_obj_module_t esp32_module = {
    .base = { &mp_type_module },
    .globals = (mp_obj_dict_t *)&esp32_module_globals,
};

MP_REGISTER_MODULE(MP_QSTR_esp32, esp32_module);

now it would be available as

import esp32

print(esp32.get_core_id())

[dcmcshan]

I added it as core_info.c in /modules and set the flag in the header file. I think that is how micropython builds, but I can just as easily put it somewhere else if your build doesn't work that way

[kdschlosser]

as far as the REPL not being made available.. It may not be.. the removal of the GIL might have something to do with that. This is the reason why I wanted to add in proper task waits instead of using a sleep. sleep should never be used in a thread.

you have to add the c file to the list of files to be compiled. That is why it is just easier to add it to the esp32 module.

this is how sleep is handled for the esp32...

void mp_hal_delay_ms(uint32_t ms) {
    uint64_t us = (uint64_t)ms * 1000ULL;
    uint64_t dt;
    uint64_t t0 = esp_timer_get_time();
    for (;;) {
        mp_handle_pending(true);
        MICROPY_PY_SOCKET_EVENTS_HANDLER
        MP_THREAD_GIL_EXIT();
        uint64_t t1 = esp_timer_get_time();
        dt = t1 - t0;
        if (dt + portTICK_PERIOD_MS * 1000ULL >= us) {
            // doing a vTaskDelay would take us beyond requested delay time
            taskYIELD();
            MP_THREAD_GIL_ENTER();
            t1 = esp_timer_get_time();
            dt = t1 - t0;
            break;
        } else {
            ulTaskNotifyTake(pdFALSE, 1);
            MP_THREAD_GIL_ENTER();
        }
    }
    if (dt < us) {
        // do the remaining delay accurately
        mp_hal_delay_us(us - dt);
    }
}

If you notice there is not any actual delay in the code. taskYIELD gets called as does ulTaskNotifyTake. The timeout is calculated as a best guess depending on how long other tasks are taking to execute. This might be getting twisted up due to you having 2 tasks running that are both "sleeping" and there is not any time that is being made available for the REPL to run properly.

That is one of the reasons I wanted to expose "events" like what is seen in CPythons threading module.

My plan was to create 2 modules. multiprocessing and threading. where multiprocessing would start a new task on the other core and threading would start a new task on the core in which the thread was created on.

example...

import multiprocess
import threading

def thread_1_core_0():
    print('thread1 running on core0')

def thread_2_core_0():
    print('thread2 running on core0')

def thread_1_core_1():
    print('thread1 running on core1')

def thread_2_core_1():
    print('thread2 running on core1')

def process_core_1():
    print('core 1 process')

    threading.Thread(target=thread_1_core_1).start()
    threading.Thread(target=thread_2_core_1).start()

multiprocess.Process(target=process_core_1).start()
threading.Thread(target=thread_1_core_0).start()
threading.Thread(target=thread_2_core_0).start()

[kdschlosser]

and sleeping us doesn't let any other tasks run at all and should only be called from the context of the main thread..

void mp_hal_delay_us(uint32_t us) {
    // these constants are tested for a 240MHz clock
    const uint32_t this_overhead = 5;
    const uint32_t pend_overhead = 150;

    // return if requested delay is less than calling overhead
    if (us < this_overhead) {
        return;
    }
    us -= this_overhead;

    uint64_t t0 = esp_timer_get_time();
    for (;;) {
        uint64_t dt = esp_timer_get_time() - t0;
        if (dt >= us) {
            return;
        }
        if (dt + pend_overhead < us) {
            // we have enough time to service pending events
            // (don't use MICROPY_EVENT_POLL_HOOK because it also yields)
            mp_handle_pending(true);
        }
    }
}

[dcmcshan]

So, I think the idea is that micropython automagically compile whatever is in modules/ but I tried it your way anyways.

Result is a little surprising - it would seem that the repl is running on core 1, which is actually what I thought micropython had decided on until you said otherwise.

Interestingly, the core0 task ends up on 0 and the core1 ends up on 1.

If I start core0 before core1 thread, both end up on core 0.

I'd be interested into insight as to why...

>>> esp32.get_core_id()
1
>>> %run -c $EDITOR_CONTENT
Starting thread_core0 on core 0
>Starting thread_core1 on core 1
Counter value:      127        2ms     0.00Hz
Counter value:     9413     1015ms  9272.91Hz
Counter value:    18709     1002ms  9275.66Hz
Counter value:    28004     1001ms  9278.99Hz
Counter value:    37300     1001ms  9280.92Hz
Counter value:    46596     1001ms  9282.07Hz
Counter value:    55891     1002ms  9281.14Hz
Counter value:    65187     1001ms  9281.93Hz

[dcmcshan]

After some experimentation, it seems that the second thread gets put on core 0, and the rest are core 1. But not always...?

[dcmcshan]

Here is the code I'm using to experiment with. Let me know when you have that threading module ready to try out, and I'll take a look.

Also, let me know what coding practices we should be using to try an be thread safe.

import _thread
import time
import esp32

# Shared counter variable
counter = 1

def thread_core1():
    print(f"\nStarting thread_core1 on core {esp32.get_core_id()}")
    global counter
    while True:
        start = time.ticks_us()
        counter += 1
        while time.ticks_diff(time.ticks_us(), start) < 100:
            pass

def print_core_and_exit():
    print(f"This thread is running on core {esp32.get_core_id()}")

t0 = time.ticks_ms()
def thread_core0():
    global t0
    print(f"\nStarting thread_core0 on core {esp32.get_core_id()}")
    start_time = time.ticks_ms()
    while True:
        current_time = time.ticks_ms()
        dt = time.ticks_diff(current_time, t0)
        t0 = current_time
        elapsed_time = time.ticks_diff(current_time, start_time) / 1000  # in seconds
        frequency = counter / elapsed_time if elapsed_time > 0 else 0  # Avoid division by zero
        print(f"Core {esp32.get_core_id()} {counter:8} {dt:8}ms {frequency:8.2f}Hz")
        time.sleep(1)

print(f"This is core {esp32.get_core_id()}")

# Start thread on Core 1
_thread.start_new_thread(thread_core1, ())

# Start the new thread that prints its core and exits
_thread.start_new_thread(print_core_and_exit, ())

# Start thread on Core 0 (main core)
_thread.start_new_thread(thread_core0, ())

[kdschlosser]

I would have to go and check specifically but I believe that you are not able to set what core to run the thread on. It is going to automatically pick the core that has the lightest amount of load on it.. That's what it does...

and you are correct about the core that both the threads and also micropython runs on...

https://github.com/micropython/micropython/blob/6835743dccf0e8d0a677e8500af0c5a4b19fe645/ports/esp32/mphalport.h#L47

[kdschlosser]

Yup it selects the core that has the lightest load..

#if (MP_USE_DUAL_CORE && !CONFIG_FREERTOS_UNICORE)
    #define _CORE_ID    tskNO_AFFINITY
#else
    #define _CORE_ID    MP_TASK_COREID
#endif

That's what tskNO_AFFINITY does.

[dcmcshan]

I tried to game this by loading trying to load the core I don't want with busy wait. I have had minimal luck. Do you know how it actually measures load? I was imagining some function of idle time... but now I am uncertain.

In this exercise, I start a busy_wait thread on core 1, and then wait a bit and then start another... hoping for the second to be on core 0, but it isn't...

Main: This is core 1
start_thread: Starting thread_core1 for core 1
Thread registry updated: {'busy_wait_1070366180': {'core': 1, 'handle': 1070366180}}
busy_wait_1070366180: Running on core 1
First busy_wait running on target core 1. Starting another.
Thread registry updated: {'busy_wait_1070371656': {'core': 1, 'handle': 1070371656}, 'busy_wait_1070366180': {'core': 1, 'handle': 1070366180}}
busy_wait_1070371656: Running on core 1
Warning: Second busy_wait not confirmed on other core. Current state: 1
wrapper: thread_core1 started on core 1
Thread registry updated: {'busy_wait_1070371656': {'core': 1, 'handle': 1070371656}, 'thread_core1': {'core': 1, 'handle': 1070377132}, 'busy_wait_1070366180': {'core': 1, 'handle': 1070366180}}
thread_core1: Running on the wrong core

[kdschlosser]

I have no idea how FreeRTOS gauges what core to place a task (thread) onto. I am working on writing a multiprocessing and a threading module that will provide a more complete CPython API for handling threads and multiple processes.

How it is going to work is this...

by default MicroPython runs on core 1. if you want to run something on core 0 you would need to create a second "process" using multiprocess.Process.. This is not a second process in reality it is simply a FreeRTOS task that points to the second core. When a thread is started it recursively looks at the thread/process that is starting the thread. It does this to know what core to run the new thread on. CPython can only create threads on the core that the parent process is running on. So the behavior will be the same.

I am going to be adding collections.Queue and multiprocess.Queue which will mallow for passing data between threads and processes. This would be only for controlling access to data as a convenience thing. The ESP32 shares memory access between the cores and the user could handle the synchronization using sephamores and locks (mutex) bt that would be a lot of added work for the user to do.

I am also going to add multiprocess.Event and threading.Event which will give the user the ability to easily stall a process/thread with a way to release that stall. This will not be a spinning wheels stall.

[dcmcshan]

I am seemingly able to consistently start a thread on core 0 with this approach. (But not core 1 - which is fine, as we can just start the thread normally), and I don't know if subsequent calls actually work...

import _thread
import time
import machine
import esp32

# Global dictionary to track threads, their cores, and handles
thread_registry = {}

def register_thread(thread_name, core, handle=None):
    global thread_registry
    thread_registry[thread_name] = {"core": core, "handle": handle}
    print(f"{'    ' if esp32.get_core_id() == 0 else ''}Thread registry updated: {thread_registry}")

def busy_wait():
    thread_id = _thread.get_ident()
    thread_name = f"busy_wait_{thread_id}"
    current_core = esp32.get_core_id()
    register_thread(thread_name, current_core, thread_id)
    print(f"{'    ' if current_core == 0 else ''}{thread_name}: Running on core {current_core}")
    while True:
        pass  # Busy wait

def start_thread(func, target_core, *args, **kwargs):
    print(f"{'    ' if esp32.get_core_id() == 0 else ''}start_thread: Starting {func.__name__} for core {target_core}")
    if target_core not in (0, 1):
        raise ValueError("Target core must be 0 or 1")

    # Start first busy_wait thread
    busy_thread_id1 = _thread.start_new_thread(busy_wait, ())
    time.sleep(0.1)  # Short delay to ensure busy_wait starts

    # Check if busy_wait is on the target core
    busy_thread_name1 = f"busy_wait_{busy_thread_id1}"
    busy_core = thread_registry.get(busy_thread_name1, {}).get("core")
    if busy_core == target_core:
        print(f"{'    ' if esp32.get_core_id() == 0 else ''}First busy_wait running on target core {target_core}. Starting another.")
        busy_thread_id2 = _thread.start_new_thread(busy_wait, ())
        time.sleep(1)  # Short delay to ensure second busy_wait starts

        # Confirm second busy_wait is on the other core
        busy_thread_name2 = f"busy_wait_{busy_thread_id2}"
        busy_core2 = thread_registry.get(busy_thread_name2, {}).get("core")
        if busy_core2 is not None and busy_core2 != target_core:
            print(f"{'    ' if esp32.get_core_id() == 0 else ''}Second busy_wait confirmed running on core {busy_core2}")
            _thread.exit()
            print(f"{'    ' if esp32.get_core_id() == 0 else ''}Delete busy_wait running on core {busy_core}")
            thread_registry.pop(busy_thread_name1, None)
        else:
            print(f"{'    ' if esp32.get_core_id() == 0 else ''}Warning: Second busy_wait not confirmed on other core. Current state: {busy_core2}")
    else:
        print(f"{'    ' if esp32.get_core_id() == 0 else ''}busy_wait confirmed running on core {busy_core}")

    time.sleep(1)

    # Start the target function
    def wrapper():
        current_core = esp32.get_core_id()
        print(f"{'    ' if current_core == 0 else ''}wrapper: {func.__name__} started on core {current_core}")
        register_thread(func.__name__, current_core, _thread.get_ident())
        func(*args, **kwargs)

    thread_handle = _thread.start_new_thread(wrapper, ())
    time.sleep(1)  # Short delay to ensure wrapper starts

    # Attempt to terminate busy_wait thread(s)
    try:
        _thread.exit()
        print(f"{'    ' if esp32.get_core_id() == 0 else ''}start_thread: Attempted to terminate busy_wait thread(s)")
        # Remove any remaining busy_wait threads from registry
        for thread_name in list(thread_registry.keys()):
            if thread_name.startswith("busy_wait_"):
                thread_registry.pop(thread_name, None)
    except Exception as e:
        print(f"{'    ' if esp32.get_core_id() == 0 else ''}start_thread: Failed to terminate busy_wait thread(s). Error: {str(e)}")

    time.sleep(0.1)  # Short delay after termination attempt

    return thread_handle

def thread_core1():
    current_core = esp32.get_core_id()
    if current_core == 1:
        print("\033[92mthread_core1: Running on the correct core\033[0m")  # Green
    else:
        print("\033[91mthread_core1: Running on the wrong core\033[0m")  # Red

def thread_core0():
    current_core = esp32.get_core_id()
    if current_core == 0:
        print("\033[92mthread_core0: Running on the correct core\033[0m")  # Green
    else:
        print("\033[91mthread_core0: Running on the wrong core\033[0m")  # Red

print(f"Main: This is core {esp32.get_core_id()}")

# Start thread on Core 0 (main core)
thread0_handle = start_thread(thread_core0, 0)

# Print final thread registry
print(f"{'    ' if esp32.get_core_id() == 0 else ''}Final thread registry: {thread_registry}")

[kdschlosser]

as I said I am writing new modules to handle this. You just have to wait patiently. I have the lock, rlock, semaphore and event portions written. I am working on the thread and process classes now.

[kdschlosser]

If you wanted to watch my progress you can view it here...

https://github.com/lvgl-micropython/lvgl_micropython/tree/threading/ext_mod

It is not ready yet but you can see how much I have gotten done.

[kdschlosser]

I wanted to give you an update. I have finished writing it and now I am working through some compile errors. Once I have those sorted out you can give it a test and lemme know if it works or if it crashes. It was a bit tricky to get to work because the threading is pretty well nested into MicroPython and almost all of it is in C source files which makes it inaccessible. With some code borrowing and adding to what was there I believe I cam up with a solution with as little modification to MicroPython as possible.

[kdschlosser]

OooOOooOOOo... I have not yet tested it but it does compile.... if you wanna give it a go I did provide all of the information needed to use it in the readme file.

you need to still supply the --dual-core-threading build argument for it to work,

[dcmcshan]

COOL!

Tried this from README as a quick test, error below. Maybe something funny in the mapping?

import threading

event = threading.Event()

def run_thread():
    i = 0
    while not event.is_set():
        print(i)
        i = i + 1
        event.wait(timeout = 1000)

t = threading.Thread(run_thread)
t.start()

# user code here...

# to have the thread exit
event.set()

Traceback (most recent call last):
  File "<stdin>", line 14, in <module>
AttributeError: 'Thread' object has no attribute 'start'

>>> help(t)
object <Thread> is of type Thread
  start -- <function>
  is_alive -- <function>

[kdschlosser]

I will take a look and see what that error is about. I could have forgotten to add it to the class.

[dcmcshan]

Can we add some utility functions to see what threads are running and what core the current thread is on?

[kdschlosser]

the way it works means there is no need to have that ability. It uses the core of the thread/process that is creating the thread. So if you create a thread from the main thread it creates the new thread using the same core the main thread is using. If you create a process and in that process you then create a thread then the thread is going to be using the core the process is using.

adding that ability is going to increase the size of the firmware when it serves no real purpose.

[dcmcshan]

Not sure I understand, but I will explore. Is there a way to assign a process to a core?

I ask because my above methodology involved loading the "other" core to get the thread where I wanted it to be...

[kdschlosser]

ok so here are some examples..

Thread that is created from the main thread is going to run on the same core as the main thread. In the case of the ESP32-S3 that is going to be core 1

import multiprocessing
import threading

def thread_cb():
    print('thread is running on core 1')

print('process is runing on core 1')
thread = threading.Thread(target=thread_cb)
thread.start()

If you create a thread from another thread it is going to run on the same core as the parent thread. In this case that is going to be core 1

import multiprocessing
import threading

def thread_cb_1():
    print('thread1 is running on core 1')
    thread2 = threading.Thread(target=thread_cb_2)
    thread2.start()

def thread_cb_2():
    print('thread2 is running on core 1')

thread1 = threading.Thread(target=thread_cb_1)
thread1.start()

To use the second core you create a process. that process is going to run on core 0. Any threads created from inside of that process will inherit the core of the process which is going to be core 0

import multiprocessing
import threading

def thread_cb():
    print('thread is running on core 0')

def process_cb():
    print('process is running on core 0')
    thread = threading.Thread(target=thread_cb)
    thread.start()

process = multiprocessing.Process(target=process_cb)
process.start()

just like with creating a thread from a thread it is going to inherit the core of that thread. so when you create a thread from a process and then a thread from that thread they are all going to use the same core which is 0.

import multiprocessing
import threading

def thread1_cb():
    print('this thread is running on core 0')
    thread2 = threading.Thread(target=thread2_cb)
    thread2.start()

def thread2_cb():
    print('thread2 is running on core 0')

def thread3_cb():
    print('thread3 is running on core 0')

def process_cb():
    print('process is running on core 0')
    thread1 = threading.Thread(target=thread1_cb)
    thread1.start()
    thread3 = threading.Thread(target=thread3_cb)
    thread3.start()

print('process is running on core 1')

process = multiprocessing.Process(target=process_cb)
process.start()

This is how CPython works. threads run on the core that the process is running on. With Python you cannot have a thread run on a different core than the process.

[dcmcshan]

Not same error as above, this one's for "Process", but likely similar fix.

Edit: I guess that same error persists - I was just using different example

process is running on core 1
Traceback (most recent call last):
  File "<stdin>", line 29, in <module>
AttributeError: 'Process' object has no attribute 'start'
>>> process
<Process>
>>> help(process)
object <Process> is of type Process
  start -- <function>
  is_alive -- <function>
>>> process.start()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Process' object has no attribute 'start'

[kdschlosser]

OK I will check it out and see what is going on.

[kdschlosser]

OK I figured out what was broke and it is now fixed.

[dcmcshan]

Fails to build.

/Users/danielmcshan/GitHub/lvgl_micropython/lib/micropython/ports/esp32/mpthreadport.c:38:10: fatal error: ../../../../ext_mod/threading/common/inc/thread_common.h: No such file or directory
38 | #include "../../../../ext_mod/threading/common/inc/thread_common.h"
|          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Not sure where this is building from, but the file does exist.

lvgl_micropython % ls ext_mod/threading/esp32/common/inc/thread_common.h 
ext_mod/threading/esp32/common/inc/thread_common.h

Edit:

 lvgl_micropython % grep -R thread_common.h
./micropy_updates/esp32/mpthreadport.c:#include "../../../../ext_mod/threading/common/inc/thread_common.h"

Not entirely sure how build is using this? I don't, in general, like to put relative paths in source code.

[dcmcshan]

Issue is that the path in mpthreadport.c is missing a /esp32.

Fixing that,

In file included from /Users/danielmcshan/GitHub/lvgl_micropython/lib/micropython/ports/esp32/mpthreadport.c:38:
/Users/danielmcshan/GitHub/lvgl_micropython/lib/micropython/ports/esp32/../../../../ext_mod/threading/esp32/common/inc/thread_common.h:20:14: fatal error: threading_thread.h: No such file or directory
20 |     #include "threading_thread.h"
|              ^~~~~~~~~~~~~~~~~~~~

At this point, I think it might make most sense to add a -I for that directory?

[kdschlosser]

the relative path is added because of the complexities of having to deal with adding an include path to the build with how the build system works in MicroPython. I want to keep the modifications to the core micropython files as small as possible and only add updates that are 100% necessary. I could have booger'd up the build script when I merged in the changes from the main branch. I may have missed something which is more than likely hr cause of what is going on. I will take a look and see what I need to do to solve the problem.

[dcmcshan]

Fixing the above `

include "../../threading/inc/threading_thread.h"`

Now gives this:

/Users/danielmcshan/GitHub/lvgl_micropython/lib/micropython/ports/esp32/../../../../ext_mod/threading/esp32/common/inc/../../threading/inc/threading_thread.h:5:10: fatal error: thread_thread.h: No such file or directory
5 | #include "thread_thread.h"
|          ^~~~~~~~~~~~~~~~~

Seems a bit circular, but I fixed that as well

`

include "../../common/inc/thread_thread.h"

`

Now I get this...

[3.a(mpthreadport.c.obj): in function `mp_thread_deinit':
/Users/danielmcshan/GitHub/lvgl_micropython/lib/micropython/ports/esp32/mpthreadport.c:263:(.text.mp_thread_deinit+0x4c): undefined reference to `threading_deinit'
collect2: error: ld returned 1 exit status](url)

[kdschlosser]

you need to give me some time to test it again and get it to compile. There are some things that I don't like how I am doing it and I am going to change them up a bit.

[kdschlosser]

OK so I have cleaned up that branch for the threading. I tested and it does compile for the ESP32. I have not tested to see if it works properly or not. If you want to give it a go you are more then welcome to.