std::thread implementation for ESP32 to Arduino API/IDE

[dsyleixa]

wishful IMO for Arduino: a std::thread implementation for ESP32, like meanwhile already available for Teensy (TeensyThread): https://github.com/ftrias/TeensyThreads or perhaps native C++ <thread>

[atanisoft]

Std::thread is usable already in arduino-esp32. Have you tried it already and found it not working?

[dsyleixa]

Std::thread is usable already in arduino-esp32. Have you tried it already and found it not working?

I don't see an Arduino .ino example anywhere for testing

• can you please provide a working .ino Example to test?

[atanisoft]

There may not be an example in the arduino-esp32 tree but there are a few in the esp-idf tree. The only change you would need to make to those is move the thread related calls from app_main() to setup(). There are also a few helper API from esp-idf to control stack size which is not part of the c++ spec.

[dsyleixa]

sorry, I don't understand you what you mean... tbh, I would need a working .ino code.

[atanisoft]

This is based on https://github.com/espressif/esp-idf/blob/master/examples/system/cpp_pthread/main/cpp_pthread.cpp with most of it stripped out so you have an example of usage. You can see more more options to configure the threads in the linked example.

#include <Arduino.h>
#include <iostream>
#include <thread>
#include <chrono>
#include <memory>
#include <string>
#include <sstream>
#include <esp_pthread.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

const auto sleep_time = seconds
{
    5
};

void print_thread_info(const char *extra = nullptr)
{
    std::stringstream ss;
    if (extra) {
        ss << extra;
    }
    ss << "Core id: " << xPortGetCoreID()
       << ", prio: " << uxTaskPriorityGet(nullptr)
       << ", minimum free stack: " << uxTaskGetStackHighWaterMark(nullptr) << " bytes.";
    Serial.println(ss.str().c_str());
}

void other_thread_func() {
    while(true) {
        print_thread_info("This is the other thread.");
        std::this_thread::sleep_for(sleep_time);
    }
}

void setup() {
    Serial.begin(115200);
    auto cfg = esp_pthread_get_default_config();
    cfg.thread_name = "other thread"; // adjust to name your thread
    cfg.stack_size = 4096; // adjust as needed
    esp_pthread_set_cfg(&cfg);
    std::thread other_thread(other_thread_func);
}
void loop() {
    print_thread_info("This is the loop() thread.");
    std::this_thread::sleep_for(sleep_time);
}

[dsyleixa]

have you tried that by your own? I tried, but it does not get compiled ( a million errors). Please try first by your own if you suggest a ino code ;)

[dsyleixa]

PS: it would be good to have an example like the following:

a) using just Arduino Serial.print(), no << stream (that doesn't understand no common Arduino user)

b) explicite variable types (auto is not common),, no sleep() but Arduino-like delay()

c) run 1 thread called void thread1() by blinking LED_BUILTIN delay(2000) and monitoring that by Serial.println("thread1: LED ON") / ("LED OFF")

d) run 1 thread called void thread2() by counting an uint_32_t counter each second by Serial.print("thread2: "); Serial.println(counter);

e) run extra thread in the main loop() by counting an uint32_t counter every 10 seconds by Serial.print("main loop : "); Serial.println(counter);

f) show how to stop threads (thread1 or thread2) prematurely and to join threads .

[atanisoft]

have you tried that by your own? I tried, but it does not get compiled ( a million errors). Please try first by your own if you suggest a ino code ;)

I will admit I did not test it, it was copied almost verbatim from ESP-IDF. There are a few problems with the code though due to arduino-esp32 including IDF 3.2. The following will do what you are asking:

#include <Arduino.h>
#include <thread>
#include <chrono>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

void counter_loop() {
    uint32_t counter = 0;
    while(true) {
        Serial.print("counter_loop: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);
    }
}

void blinker_loop() {
    uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        std::this_thread::sleep_for(one_sec);
        digitalWrite(LED_BUILTIN, LOW);
        std::this_thread::sleep_for(one_sec);
    }
}

void setup() {
    Serial.begin(115200);
    pinMode(LED_BUILTIN, OUTPUT);
    std::thread counter_loop_thread(counter_loop);
    std::thread blinker_loop_thread(blinker_loop);
}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.print("main loop: ");
    Serial.println(main_loop_counter);
    delay(10000);
}

As for stopping a thread, you will need to refer to the C++ spec or add some sort of global flag that the threads will check to see if they should keep running. That is not specific to arduino-esp32 and would likely be better answered on the forums.

[dsyleixa]

thank you, it can be compiled and uploaded, but lots of runtime errors:

Rebooting... ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:928 ho 0 tail 12 room 4 load:0x40078000,len:8424 ho 0 tail 12 room 4 load:0x40080400,len:5868 entry 0x4008069c counter_loop: 0 abort() was called at PC 0x400e5ca7 on core 1

Backtrace: 0x40089150:0x3ffb1ed0 0x4008937d:0x3ffb1ef0 0x400e5ca7:0x3ffb1f10 0x400e5cee:0x3ffb1f30 0x400d0da2:0x3ffb1f50 0x400d0f92:0x3ffb1f70 0x400d1bc3:0x3ffb1fb0 0x40087c9d:0x3ffb1fd0

Rebooting... ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:928 ho 0 tail 12 room 4 load:0x40078000,len:8424 ho 0 tail 12 room 4 load:0x40080400,len:5868 entry 0x4008069c counter_loop: 0 abort() was called at PC 0x400e5ca7 on core 1

(repeatedly)

so again, please test by yourself before posting it 8)

[atanisoft]

so again, please test by yourself before posting it 8)

I'm only providing a sample for you to start with, you can decode the stacktrace and go from there. I'm not using this functionality myself or continuing to work on this example.

[dsyleixa]

sorry, I appreciate your input, but a really working example is needed, for common use to common Arduino users. I personally don't know C++ std::thread at all, I am only a little (minor) experienced in POSIX C99 pthread, and most of the Arduino users do not even know THAT ;)

That's why this (working) functionality is needed, for educational reasons and purposes 8)

[atanisoft]

but a really working example is needed, for common use to common Arduino users.

There is a working example in the IDF tree, but until IDF v3.3 is picked up by arduino-esp32 (after 1.0.3 likely) it is unlikely it will be meaningful here.

I personally don't know C++ std::thread at all, I am only a little (minor) experienced in POSIX C99 pthread, and most of the Arduino users do not even know THAT ;)

std::thread is based on pthread to an extent but there are limitations built into the std::thread specification, thread stack size is one area that was omitted from the specification. There are multiple options on how to work around this though, esp_pthread_get_default_config() being one that is available in IDF v3.3 or later.

That's why this (working) functionality is needed, for educational reasons and purposes 8)

This is also why the forums are ideal for this and not an issue on here.

[dsyleixa]

I have to disagree, my point is that I am requesting a working example code for Arduino, like the ones which already exist for WifiServer or WebServer or SD or SPIFFS or whatever, integrated into the standard Arduino IDE examples OTOH, I daresay that my recent topics in the esp32 forum have not be resolved at all, in the absence of participation and content proposals, but again: not discussion is my intent, but a provided working Arduino API example.

[stickbreaker]

@dsyleixa

Wow, "Do it for me, I'm a Snow Flake and Can't do it myself."

You do realize this is a volunteer project? No one owes you anything? If you are dissatisfied with the level of support you are receiving, I will provide fee based support, my current professional rates are $200/hr with 10 hr minimum. For more complex projects monthly or annual contracts are possible.

Chuck.

[dsyleixa]

@dsyleixa

Wow, "Do it for me, I'm a Snow Flake and Can't do it myself."

You do realize this is a volunteer project? No one owes you anything? If you are dissatisfied with the level of support you are receiving, I will provide fee based support, my current professional rates are $200/hr with 10 hr minimum. For more complex projects monthly or annual contracts are possible.

Chuck.

I prefer free support, like I am used to on Arduino platform. What I pay for is lots of hardware though, as far as I am getting free support for it. Arduino is designed for hobbyists and artists not having the programming skills like professional computer scientists. That's finally the reason for world-wide tremendous success of the Arduino project.

[atanisoft]

I prefer free support, like I am used to on Arduino platform.

free support is one thing, asking someone to write something from scratch for you is another. I've provided you an example to get you started. You then said the program crashed and you provided a stacktrace for it, unfortunately the stacktrace is meaningless without being decoded (the issue template links to how to decode it for inclusion in the issue). If you want to use std::thread in your arduino-esp32 programs it is time to dig into the std::thread reference documentation on https://en.cppreference.com/w/cpp/thread/thread which also provides examples that should work as written on the esp32.

[dsyleixa]

I have no idea what the error messages mean and what causes the repeated reboots, it seems that the esp32 cannot process the std::threads in the way you (or the esp libs) implemented them so it can't run that code accordingly.

[dsyleixa]

update: thanks to 2 members of a different (German) forum the code could be fixed - mainly std::thread counter_loop_thread(counter_loop); std::thread blinker_loop_thread(blinker_loop); had to be made global!

(besides that, a counter++ each was missing )

feel free to copy and use that for the official esp32 repo examples!

[CODE]

#include <Arduino.h>
#include <thread>
#include <chrono>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

void counter_loop() {
    uint32_t counter = 0;
    while(true) {
        Serial.print("counter_loop: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);
        counter++;
    }
}

void blinker_loop() {
    uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.print("blinker_loop (HIGH) counter: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.print("blinker_loop (LOW) counter: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

std::thread counter_loop_thread(counter_loop);
std::thread blinker_loop_thread(blinker_loop);

void setup() {
    Serial.begin(115200);
    pinMode(LED_BUILTIN, OUTPUT);

}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.print("main loop: ");
    Serial.println(main_loop_counter);
    delay(10000);
}

[/CODE]

[QUOTE]

main loop: 1 counter_loop: 1 blinker_loop (LOW) counter: 0 blinker_loop (HIGH) counter: 1 counter_loop: 2 counter_loop: 3 blinker_loop (LOW) counter: 1 blinker_loop (HIGH) counter: 2 counter_loop: 4 counter_loop: 5 blinker_loop (LOW) counter: 2 blinker_loop (HIGH) counter: 3 counter_loop: 6 counter_loop: 7 blinker_loop (LOW) counter: 3 blinker_loop (HIGH) counter: 4 counter_loop: 8 counter_loop: 9 blinker_loop (LOW) counter: 4 blinker_loop (HIGH) counter: 5 counter_loop: 10 main loop: 2 counter_loop: 11 blinker_loop (LOW) counter: 5 blinker_loop (HIGH) counter: 6 counter_loop: 12 counter_loop: 13 blinker_loop (LOW) counter: 6 blinker_loop (HIGH) counter: 7 counter_loop: 14 counter_loop: 15 blinker_loop (LOW) counter: 7 blinker_loop (HIGH) counter: 8 counter_loop: 16 counter_loop: 17 blinker_loop (LOW) counter: 8 blinker_loop (HIGH) counter: 9 counter_loop: 18 counter_loop: 19 blinker_loop (LOW) counter: 9 blinker_loop (HIGH) counter: 10 counter_loop: 20 main loop: 3 counter_loop: 21

[/QUOTE]

A perfect base to start from for further applications! A big leap for mankind! 8) thanks to all, especially @atanisoft !

[dsyleixa]

edit, update: interestingly, unexpectedly, the led_blinker_loop counter is incremeted between LED ON/OFF-pairs, although it's in the same thread loop by identical counter state, and the inc had been expected just for either future LED ON/OFF pairs each then.... (?)

[atanisoft]

I suspect the reason those std::thread objects needed to be global is so they don't go out of scope when setup() completes. Another (and better) way to fix this would be to declare them as std::unique_ptr in global scope and allocate them from setup() so they are guaranteed to start when you expect them to.

As for the odd behavior on the blinker counter, you can observe the same for counter_loop as well. It skips 0 in both cases. I suspect this may be due to global declaration and the thread starting before Serial.begin() has executed, but that is only a guess. It could be something else.

[dsyleixa]

thank you, as to the thread declarations, do you mean that way...?

std::unique_ptrstd::thread  counter_loop_thread(counter_loop);
std::unique_ptrstd::thread  blinker_loop_thread(blinker_loop);

as to the odd counters: how could that be fixed? it looks as if that is a bug in the libs or the esp32 core? After all, this odd behaviour is not logical, is it?

[atanisoft]

as to the thread declarations, do you mean that way...?

close, try this:

std::unique_ptr<std::thread>  counter_loop_thread;
std::unique_ptr<std::thread>  blinker_loop_thread;
void setup() {
  Serial.begin(115200L);
  counter_loop_thread.reset(new std::thread(counter_loop));
  blinker_loop_thread.reset(new std::thread(blinker_loop));
}

This effectively splits the declaration and allocation into two distinct steps using C++ notation. It can also be done as:

std::thread *counter_loop_thread;
std::thread *blinker_loop_thread;
void setup() {
  Serial.begin(115200L);
  counter_loop_thread = new std::thread(counter_loop);
  blinker_loop_thread = new std::thread(blinker_loop);
}

You can even go further to use a llamda function in place of counter_loop or blinker_loop if you prefer that style instead.

[dsyleixa]

thank you, that looks fine, especially as to the counters in the blink_loop (AFAICS):

counter_loop: 0 blinker_loop (HIGH) counter: 0 main loop: 1 blinker_loop (LOW) counter: 0 counter_loop: 1 counter_loop: 2 blinker_loop (HIGH) counter: 1 blinker_loop (LOW) counter: 1 counter_loop: 3 counter_loop: 4 blinker_loop (HIGH) counter: 2 blinker_loop (LOW) counter: 2 counter_loop: 5 counter_loop: 6 blinker_loop (HIGH) counter: 3 blinker_loop (LOW) counter: 3 counter_loop: 7 counter_loop: 8 blinker_loop (HIGH) counter: 4 blinker_loop (LOW) counter: 4 counter_loop: 9 counter_loop: 10 blinker_loop (HIGH) counter: 5 main loop: 2 blinker_loop (LOW) counter: 5 counter_loop: 11 counter_loop: 12 blinker_loop (HIGH) counter: 6 blinker_loop (LOW) counter: 6 counter_loop: 13 counter_loop: 14 blinker_loop (HIGH) counter: 7 blinker_loop (LOW) counter: 7 counter_loop: 15 counter_loop: 16 blinker_loop (HIGH) counter: 8 blinker_loop (LOW) counter: 8 counter_loop: 17 counter_loop: 18 blinker_loop (HIGH) counter: 9 blinker_loop (LOW) counter: 9 counter_loop: 19 counter_loop: 20 blinker_loop (HIGH) counter: 10 main loop: 3 blinker_loop (LOW) counter: 10

code:

// std::thread for ESP32, Arduino IDE

#include <Arduino.h>
#include <thread>
#include <chrono>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

void counter_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        Serial.print("counter_loop: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);
        counter++;
    }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.print("blinker_loop (HIGH) counter: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.print("blinker_loop (LOW) counter: ");
        Serial.println(counter);
        std::this_thread::sleep_for(one_sec);
        counter++;
    }
}

std::unique_ptr<std::thread>  counter_loop_thread;
std::unique_ptr<std::thread>  blinker_loop_thread;

void setup() {
  Serial.begin(115200);
  //debug
  delay(2000);

  counter_loop_thread.reset(new std::thread(counter_loop));
  blinker_loop_thread.reset(new std::thread(blinker_loop));
}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.print("main loop: ");
    Serial.println(main_loop_counter);
    delay(10000);
}

[dsyleixa]

a question to understanding: can std::this_thread::sleep_for(one_sec); // 1 second always be exchanged by delay(1000); // 1000 milliseconds :?:

[atanisoft]

Possibly, it would depend on the implementation of the std::thread bindings provided by ESP-IDF. I haven't looked at them specifically but if you are using std::thread I'd suggest stick with just it's declared model and not mix in Arduino APIs where there is already an API provided by std::thread.

[dsyleixa]

ok, I'll try out...

1st, your alternative code from above also works like the former one:

std::thread *counter_loop_thread;
std::thread *blinker_loop_thread;

void setup() {
  Serial.begin(115200);
  //debug
  delay(2000);
  counter_loop_thread = new std::thread(counter_loop);
  blinker_loop_thread = new std::thread(blinker_loop);
}

counter_loop: 0 blinker_loop (HIGH) counter: 0 main loop: 1 blinker_loop (LOW) counter: 0 counter_loop: 1 counter_loop: 2 blinker_loop (HIGH) counter: 1 blinker_loop (LOW) counter: 1 counter_loop: 3 counter_loop: 4 blinker_loop (HIGH) counter: 2 blinker_loop (LOW) counter: 2 counter_loop: 5 counter_loop: 6 blinker_loop (HIGH) counter: 3 blinker_loop (LOW) counter: 3 counter_loop: 7 counter_loop: 8 blinker_loop (HIGH) counter: 4 blinker_loop (LOW) counter: 4 counter_loop: 9 counter_loop: 10 blinker_loop (HIGH) counter: 5 main loop: 2 blinker_loop (LOW) counter: 5 counter_loop: 11 counter_loop: 12 blinker_loop (HIGH) counter: 6 blinker_loop (LOW) counter: 6 counter_loop: 13 counter_loop: 14 blinker_loop (HIGH) counter: 7 blinker_loop (LOW) counter: 7 counter_loop: 15 counter_loop: 16 blinker_loop (HIGH) counter: 8 blinker_loop (LOW) counter: 8 counter_loop: 17 counter_loop: 18 blinker_loop (HIGH) counter: 9 blinker_loop (LOW) counter: 9 counter_loop: 19 counter_loop: 20 blinker_loop (HIGH) counter: 10 main loop: 3 blinker_loop (LOW) counter: 10

... to be continued...

[atanisoft]

ok, I'll try out...

1st, your alternative code from above also works like the former one:

Yes, they are functionally equivalent in this very simple code example. However, this is not always the case, there are also a lot of advantages to std::unique_ptr over a raw pointer declaration. But those are unrelated to the esp32.

[dsyleixa]

ok, so in case it's equivalent at runtime then this 2nd version appears to be more handy and more intuitively understandable to "common Arduino users"

---

now 2nd, exchanging the std sleeps by delay(1000) seem to work as well, AFAICS:

counter_loop: 0 blinker_loop (HIGH) counter: 0 main loop: 1 blinker_loop (LOW) counter: 0 counter_loop: 1 counter_loop: 2 blinker_loop (HIGH) counter: 1 blinker_loop (LOW) counter: 1 counter_loop: 3 counter_loop: 4 blinker_loop (HIGH) counter: 2 blinker_loop (LOW) counter: 2 counter_loop: 5 counter_loop: 6 blinker_loop (HIGH) counter: 3 blinker_loop (LOW) counter: 3 counter_loop: 7 counter_loop: 8 blinker_loop (HIGH) counter: 4 blinker_loop (LOW) counter: 4 counter_loop: 9 counter_loop: 10 blinker_loop (HIGH) counter: 5 main loop: 2 blinker_loop (LOW) counter: 5 counter_loop: 11 counter_loop: 12 blinker_loop (HIGH) counter: 6 blinker_loop (LOW) counter: 6 counter_loop: 13 counter_loop: 14 blinker_loop (HIGH) counter: 7 blinker_loop (LOW) counter: 7 counter_loop: 15 counter_loop: 16 blinker_loop (HIGH) counter: 8 blinker_loop (LOW) counter: 8 counter_loop: 17 counter_loop: 18 blinker_loop (HIGH) counter: 9 blinker_loop (LOW) counter: 9 counter_loop: 19 counter_loop: 20 blinker_loop (HIGH) counter: 10 main loop: 3 blinker_loop (LOW) counter: 10

// std::thread for ESP32, Arduino IDE

#include <Arduino.h>
#include <thread>
#include <chrono>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

void counter_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        Serial.print("counter_loop: ");
        Serial.println(counter);
        delay(1000);  
        counter++;
    }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.print("blinker_loop (HIGH) counter: ");
        Serial.println(counter);
        delay(1000);  

        digitalWrite(LED_BUILTIN, LOW);
        Serial.print("blinker_loop (LOW) counter: ");
        Serial.println(counter);

        delay(1000);  
        counter++;
    }
}

std::thread *counter_loop_thread;
std::thread *blinker_loop_thread;

void setup() {
  Serial.begin(115200);
  //debug
  delay(2000);
  counter_loop_thread = new std::thread(counter_loop);
  blinker_loop_thread = new std::thread(blinker_loop);
}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.print("main loop: ");
    Serial.println(main_loop_counter);
    delay(10000);
}

• but IMO, for a public usage that delay() behaviour should preferably be tested by specialists upon thread-safety and race conditions though, what would you think?

[atanisoft]

ok, so in case it's equivalent at runtime then this 2nd version appears to be more handy and more intuitively understandable to "common Arduino users"

Maybe, but it is unlikely what will be accepted as part of a demo. There are a number of usages of C++ style pointer templates already in use.

• but IMO, for a public usage that delay() behaviour should preferably be tested by specialists upon thread-safety and race conditions though, what would you think?

It may work this time but there is absolutely no guarantee on safety over the long term. As mentioned above, mixing the Arduino API where a C++ API exists for that purpose should be avoided to ensure everything works as expected under all cases.

[dsyleixa]

ok, I see, gladly looking forward to a publication as an Arduino ESP32 std::thread example! thanks a lot for your contributions, and I'd be glad if I could help testing that.

[dsyleixa]

slight refinement: to make it didactical more clear which one a thread name is vs. a function name, and that both names don't depend on each other, I changed the thread names like following:

std::thread *thread_1;
std::thread *thread_2;

void setup() {
  Serial.begin(115200);
  //debug
  delay(2000);
  thread_1 = new std::thread(counter_loop);
  thread_2 = new std::thread(blinker_loop);
}

HTH!

[dsyleixa]

PS, Arduino.cc people are extremely restrained as to further documentation to delay() in MT environments, (IIUC) mostly because MT is not very much supported at all yet. OTOH, both delay() and millis() and microseconds() are the most common and basic Arduino API functions at all, widely used in almost either ino code, and so it would be highly appreciated if esp32-Arduino-APIs ensure to be (downwards) compatible to these Arduino basics.

[dsyleixa]

just observed a new issue about this std::thread behaviour:

after changing the ode of 1 thread (->fibonacci), the main loop counter is no longer printed on Serial: (both for
delay(5000); and for std::this_thread::sleep_for(5*one_sec); )

// std::thread for ESP32, Arduino IDE
// ver 0.0.5 fibonacci

#include <Arduino.h>
#include <thread>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

int fibbonacci(int n) {
   if(n == 0){
      return 0;
   } else if(n == 1) {
      return 1;
   } else {
      return (fibbonacci(n-1) + fibbonacci(n-2));
   }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println((String)"blinker_loop (HIGH) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.println((String)"blinker_loop (LOW) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    while(true) {
        for(i=30; i<41; i++) {    // limits: test, debug
          Serial.println( (String)"Fibbonacci of "+i+"="+fibbonacci(i));            
        }        
        Serial.println((String)"fibonacci_loop counter: "+counter);  
        counter++;      
    }
}

std::thread *thread_1;
std::thread *thread_2;

void setup() {
  Serial.begin(115200);
  //debug
  delay(1000);
  thread_1 = new std::thread(blinker_loop);
  thread_2 = new std::thread(fibonacci_loop);
}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.println((String)"main loop: " + main_loop_counter);
    //std::this_thread::sleep_for(5*one_sec);
    delay(5000);
}

output:

blinker_loop (HIGH) counter: 0 Fibbonacci of 30=832040 Fibbonacci of 31=1346269 Fibbonacci of 32=2178309 blinker_loop (LOW) counter: 0 Fibbonacci of 33=3524578 blinker_loop (HIGH) counter: 1 Fibbonacci of 34=5702887 blinker_loop (LOW) counter: 1 blinker_loop (HIGH) counter: 2 Fibbonacci of 35=9227465 blinker_loop (LOW) counter: 2 blinker_loop (HIGH) counter: 3 Fibbonacci of 36=14930352 blinker_loop (LOW) counter: 3 blinker_loop (HIGH) counter: 4 blinker_loop (LOW) counter: 4 blinker_loop (HIGH) counter: 5 Fibbonacci of 37=24157817 blinker_loop (LOW) counter: 5 blinker_loop (HIGH) counter: 6 blinker_loop (LOW) counter: 6 blinker_loop (HIGH) counter: 7 blinker_loop (LOW) counter: 7 blinker_loop (HIGH) counter: 8 blinker_loop (LOW) counter: 8 Fibbonacci of 38=39088169 blinker_loop (HIGH) counter: 9 blinker_loop (LOW) counter: 9 blinker_loop (HIGH) counter: 10 blinker_loop (LOW) counter: 10 blinker_loop (HIGH) counter: 11 blinker_loop (LOW) counter: 11 blinker_loop (HIGH) counter: 12 blinker_loop (LOW) counter: 12 blinker_loop (HIGH) counter: 13 blinker_loop (LOW) counter: 13 blinker_loop (HIGH) counter: 14 blinker_loop (LOW) counter: 14 Fibbonacci of 39=63245986 blinker_loop (HIGH) counter: 15 blinker_loop (LOW) counter: 15 blinker_loop (HIGH) counter: 16 blinker_loop (LOW) counter: 16 blinker_loop (HIGH) counter: 17 blinker_loop (LOW) counter: 17 blinker_loop (HIGH) counter: 18 blinker_loop (LOW) counter: 18 blinker_loop (HIGH) counter: 19 blinker_loop (LOW) counter: 19 blinker_loop (HIGH) counter: 20 blinker_loop (LOW) counter: 20 blinker_loop (HIGH) counter: 21 blinker_loop (LOW) counter: 21 blinker_loop (HIGH) counter: 22 blinker_loop (LOW) counter: 22 blinker_loop (HIGH) counter: 23 blinker_loop (LOW) counter: 23 Fibbonacci of 40=102334155 fibonacci_loop counter: 0 Fibbonacci of 30=832040 Fibbonacci of 31=1346269 blinker_loop (HIGH) counter: 24 Fibbonacci of 32=2178309 Fibbonacci of 33=3524578 blinker_loop (LOW) counter: 24 Fibbonacci of 34=5702887 blinker_loop (HIGH) counter: 25 blinker_loop (LOW) counter: 25 Fibbonacci of 35=9227465 blinker_loop (HIGH) counter: 26 blinker_loop (LOW) counter: 26 Fibbonacci of 36=14930352 blinker_loop (HIGH) counter: 27 blinker_loop (LOW) counter: 27 blinker_loop (HIGH) counter: 28 blinker_loop (LOW) counter: 28 blinker_loop (HIGH) counter: 29 Fibbonacci of 37=24157817 blinker_loop (LOW) counter: 29 blinker_loop (HIGH) counter: 30 blinker_loop (LOW) counter: 30

[atanisoft]

just observed a new issue about this std::thread behaviour:

after changing the ode of 1 thread (->fibonacci), the main loop counter is no longer printed on Serial: (both for delay(5000); and for std::this_thread::sleep_for(5*one_sec); )

This could be due to the default configuration of threads created via std::thread, the default priority is 5 which is higher than the loop() method runs at. So if you do not have a delay/yield the FreeRTOS task scheduler will not switch to other tasks. I suspect that the std::this_thread::delay_for() may not be using the FreeRTOS vTaskDelay API, but the Arduino API delay/delayMicroseconds does use this internally.

You can see the default configuration for std::thread here: https://github.com/espressif/arduino-esp32/blob/master/tools/sdk/sdkconfig#L743-L744

You can adjust this before creating the std::thread instance by using methods available in https://github.com/espressif/arduino-esp32/blob/master/tools/sdk/include/pthread/esp_pthread.h (esp_pthread_set_cfg).

[dsyleixa]

thank you, but although I see

typedef struct {
    size_t stack_size;    ///< the stack size of the pthread
    size_t prio;          ///< the thread's priority
    bool inherit_cfg;     ///< inherit this configuration further
} esp_pthread_cfg_t;

I'm afraid I don't see how to call that anywhere for which special prio and thread adjustement in my ino program regrettably.

[atanisoft]

something like this possibly:

 esp_pthread_cfg_t cfg;
 esp_pthread_get_cfg(&cfg);
 cfg.prio=1;
 esp_pthread_set_cfg(&cfg);
 thread_1 = new std::thread(blinker_loop);

[dsyleixa]

error: exit status 1 'esp_pthread_cfg_t' was not declared in this scope

void setup() {
  Serial.begin(115200);
  //debug
  delay(1000);

  esp_pthread_cfg_t cfg;
  esp_pthread_get_cfg(&cfg);
  cfg.prio=3;
  esp_pthread_set_cfg(&cfg);

  thread_1 = new std::thread(blinker_loop);
  thread_2 = new std::thread(fibonacci_loop);
}

[atanisoft]

You didn't include the required header that defines the function or the data type. Also priority 3 is higher than loop() runs at (which is 1)

[dsyleixa]

ok, I see, thx.... unfortunately the main loop prints are still not visible, I'm afraid

// std::thread for ESP32, Arduino IDE

// ver 0.0.5 fibonacci

#include <Arduino.h>
#include <thread>
#include <esp_pthread.h>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

int fibbonacci(int n) {
   if(n == 0){
      return 0;
   } else if(n == 1) {
      return 1;
   } else {
      return (fibbonacci(n-1) + fibbonacci(n-2));
   }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println((String)"blinker_loop (HIGH) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.println((String)"blinker_loop (LOW) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    while(true) {
        for(i=30; i<41; i++) {    // limits: test, debug
          Serial.println( (String)"Fibbonacci of "+i+"="+fibbonacci(i));            
        }        
        Serial.println((String)"fibonacci_loop counter: "+counter);  
        counter++;      
    }
}

std::thread *thread_1;
std::thread *thread_2;

void setup() {
  Serial.begin(115200);
  //debug
  delay(1000);

  esp_pthread_cfg_t cfg;
  esp_pthread_get_cfg(&cfg);
  cfg.prio=1;
  esp_pthread_set_cfg(&cfg);

  thread_1 = new std::thread(blinker_loop);
  thread_2 = new std::thread(fibonacci_loop);
}

uint32_t main_loop_counter = 0;
void loop() {
    main_loop_counter++;
    Serial.println((String)"main loop: " + main_loop_counter);
    //std::this_thread::sleep_for(5*one_sec);
    delay(5000);
}

blinker_loop (HIGH) counter: 0 Fibbonacci of 30=832040 Fibbonacci of 31=1346269 Fibbonacci of 32=2178309 blinker_loop (LOW) counter: 0 Fibbonacci of 33=3524578 blinker_loop (HIGH) counter: 1 Fibbonacci of 34=5702887 blinker_loop (LOW) counter: 1 blinker_loop (HIGH) counter: 2 Fibbonacci of 35=9227465 blinker_loop (LOW) counter: 2 blinker_loop (HIGH) counter: 3 Fibbonacci of 36=14930352 blinker_loop (LOW) counter: 3 blinker_loop (HIGH) counter: 4 blinker_loop (LOW) counter: 4 blinker_loop (HIGH) counter: 5 Fibbonacci of 37=24157817 blinker_loop (LOW) counter: 5 blinker_loop (HIGH) counter: 6 blinker_loop (LOW) counter: 6 blinker_loop (HIGH) counter: 7 blinker_loop (LOW) counter: 7 blinker_loop (HIGH) counter: 8 blinker_loop (LOW) counter: 8 Fibbonacci of 38=39088169 blinker_loop (HIGH) counter: 9 blinker_loop (LOW) counter: 9 blinker_loop (HIGH) counter: 10 blinker_loop (LOW) counter: 10 blinker_loop (HIGH) counter: 11 blinker_loop (LOW) counter: 11 blinker_loop (HIGH) counter: 12 blinker_loop (LOW) counter: 12 blinker_loop (HIGH) counter: 13 blinker_loop (LOW) counter: 13 blinker_loop (HIGH) counter: 14 blinker_loop (LOW) counter: 14 Fibbonacci of 39=63245986 blinker_loop (HIGH) counter: 15 blinker_loop (LOW) counter: 15 blinker_loop (HIGH) counter: 16 blinker_loop (LOW) counter: 16 blinker_loop (HIGH) counter: 17 blinker_loop (LOW) counter: 17 blinker_loop (HIGH) counter: 18 blinker_loop (LOW) counter: 18 blinker_loop (HIGH) counter: 19 blinker_loop (LOW) counter: 19 blinker_loop (HIGH) counter: 20 blinker_loop (LOW) counter: 20 blinker_loop (HIGH) counter: 21 blinker_loop (LOW) counter: 21 blinker_loop (HIGH) counter: 22 blinker_loop (LOW) counter: 22 blinker_loop (HIGH) counter: 23 blinker_loop (LOW) counter: 23 Fibbonacci of 40=102334155 fibonacci_loop counter: 0 Fibbonacci of 30=832040 Fibbonacci of 31=1346269 blinker_loop (HIGH) counter: 24 Fibbonacci of 32=2178309 Fibbonacci of 33=3524578 blinker_loop (LOW) counter: 24 Fibbonacci of 34=5702887 blinker_loop (HIGH) counter: 25 blinker_loop (LOW) counter: 25 Fibbonacci of 35=9227465 blinker_loop (HIGH) counter: 26 blinker_loop (LOW) counter: 26 Fibbonacci of 36=14930352

[atanisoft]

without some sort of delay in the fibbonacci loop you won't see any output from loop() as it doesn't yield to the other tasks that need to run as well.

[dsyleixa]

I acually chose exactly that setup without any delay in 1 thread to test for preemptivity. If delays are needed to switch the scheduler timeslice, then it's just cooperative, I'm afraid...

But is it perhaps an option to run main loop at prio 4 by default and choose prio <4 for std::threads?

[atanisoft]

But is it perhaps an option to run main loop at prio 4 by default and choose prio <4 for std::threads?

Unless you switch to using arduino-esp32 as an IDF component you will have no control over the priority of the loopTask.

I acually chose exactly that setup without any delay in 1 thread to test for preemptivity. If delays are needed to switch the scheduler timeslice, then it's just cooperative, I'm afraid...

This is scheduler dependent and there are no guarantees that it will switch tasks as you might hope/expect. You will need to test and confirm that it behaves as you expect.

[dsyleixa]

Unless you switch to using arduino-esp32 as an IDF component you will have no control over the priority of the loopTask.

But I tried that, lowering the threads to min prio 1. Having done that already it is not reasonable that a thread still blocks main loop() forcibly absolutely completely.

[lbernstone]

Just to make things clear for someone dropping in on this conversation, std::thread is NOT the recommended method for creating tasks in esp32. FreeRTOS is available which provides full facilities for tasks, events, mutexes and semaphores.

[dsyleixa]

well, this topic is about implementing C++ std::thread accordingly, and C++ <thread> is supposed to be a valid and approved standard library, also providing events, mutexes, semaphores, and more .

[dsyleixa]

so just to return to topic, having lowered the threads to min prio 1, it is not reasonable that a thread still blocks main loop() forcibly absolutely completely. And just to say, C++ std::thread is not cooperative but preemptive (what is crucial), like POSIX pthread.

[atanisoft]

And just to say, C++ std::thread is not cooperative but preemptive (what is crucial), like POSIX pthread.

That would be because std::thread is based on pthread in virtually every implementation.

[dsyleixa]

yes, I appreciate that, I always have been fine with pthread on my Raspi, it runs like a charm.

[stickbreaker]

@dsyleixa 1>0, 2>1 If a thread of higher priority is ready, it gets time. lower priority threads or equal priority will stall UNTIL ALL higher priority Thread are satisfied. If you have three threads ready at priority 3, ONLY two of those thread will execute because ESP32 has two cores period. UNTIL one of the executing priority 3 threads stall the third priority 3 thread does not get any time. There is no fairness/round robin scheduler. Pure priority. When you code threads you take the responsibility to share resources. A good thread executes a short task, releases and lets others execute. A continuous task needs to exist in loop(). Tasks should be considered background operations

Chuck.

[dsyleixa]

I know about priorities and sleep/delays from my pthreads on my Raspi Linux. As to esp32, it would be fine and reasonable to have e.g. the following thread environment setups:

a) either all threads (4,5,6...) incl. main loop() running at the same priority (e.g., 3), and then the scheduler is expected to split the time slice resources equally (even if there are no delays at all in either thread)

b) or have 1 thread without sleep/delays run at prio 2 and 4 threads running at prio 3 providing internal sleeps/delays: the scheduler is expected to split the time slice resources between the 4 prio-3-threads equally and in between running the prio-2-thread during avalible intermediate sleep periods

c) of course, having 4 prio-3-threads without sleep/delays and 1 extra prio 2 thread, then the prio 2 thread would have no chance.

All that is supposed to run that way not only on a 4core Raspi but also on a single core Raspi A or B or Zero, no multiple cores needed, and also the same way on an ESP32.

Anyway, but the crucial issue is: Currently the loop in main has no chance at all, that is not reasonable, as all std::threads have just prio 1. So the main loop prio has to become increased to a higher thread level.

admittedly I do not understand your statement about 2core ESP32, finally preemptive multithreading is not multitasking on different cores: one could have dozens of threads running on 1 core by preemptive timeslice scheduling (and equal prios, no sleeps in between).