std::thread implementation for ESP32 to Arduino API/IDE

[dsyleixa]

wishful IMO for Arduino: a std::thread implementation for ESP32, like meanwhile already available for Teensy (TeensyThread): https://github.com/ftrias/TeensyThreads or perhaps native C++ <thread>

[atanisoft]

That would run also on a single Raspi A or B or Zero, no multiple cores needed.

rPi runs Linux so it has a preemptive kernel. The ESP32 does not run Linux so it is not a comparable platform, regardless of number of cores.

But currently the loop in main has no chance at all, that is not reasonable, as all std::threads have just prio 1. So the main loop prio has to become increased to a higher std:thread level.

As stated previously, unless you use arduino-esp32 as an IDF component you have no control over the priority of the loop task. However, you have full control over the priorities for std::thread IF you call esp_pthread_set_cfg prior to creating the std::thread instance, the std::thread specification does not cover stack size or priority so it is left to the implementation to decide how to declare these.

a) either all threads (4,5,6...) incl. main loop() running at the same priority (e.g., 3), and then the scheduler is expected to split the time slice resources equally (even if there are no delays at all in either thread)

The FreeRTOS task scheduler is not a preemptive scheduler, it is a co-operative scheduler.

b) or have 1 thread without sleep/delays run at prio 2 and 4 threads running at prio 3 providing internal sleeps/delays: the scheduler is expected to split the time slice resources between the 4 prio-3-threads equally and in between running the prio-2-thread during avalible intermediate sleep periods

The FreeRTOS scheduler will rotate between the highest priority tasks that are ready to run, when the highest priority tasks are not ready to run it will look at lower priority tasks etc. Tasks can also be bound to run only on one core (loopTask always runs on core 1) so it is possible that tasks that could run will be skipped as their "bound core" is actively running something else.

[dsyleixa]

what s a IDF component? I'm running an Arduino program in the Arduino IDE with Arduino API functions, and this topic is about preemptive std::thread in this environment.

The FreeRTOS task scheduler is not a preemptive scheduler, it is a co-operative scheduler.

If RTOS is just cooperative, then it's the wrong way to use it: std.:threads needs a preemptive scheduler.

However, you have full control over the priorities for std::thread IF you call esp_pthread_set_cfg prior to creating the std::thread instance, the std::thread specification does not cover stack size or priority so it is left to the implementation to decide how to declare these.

I did that already, I decreased the thread prios to 1, nonetheless main loop() is completely, absolutely blocked.

[atanisoft]

If RTOS is just cooperative, then it's the wrong way to use it: std.:threads needs a preemptive scheduler.

And this is why it is not the recommended approach. FreeRTOS is used by ESP-IDF which everything else is built upon, including the pthread wrappers.

what s a IDF component? I'm running an Arduino program in the Arduino IDE with Arduino API functions, and this topic is about preemptive std::thread in this environment.

ESP-IDF is the underlying stack for the arduino-esp32 code, it does NOT run natively inside the Arduino IDE. It requires a different build environment that can be a lot more involved for Arduino IDE users typically will use.

[dsyleixa]

so IIUC, the preemptive scheduler is broken or defective or incomplete for Arduino? Then the Arduino implementation is faulty and thus has to be changed to the full IDF thing. preemptive MT is crucial for std::thread, either which extra libs it will demand, it has to be fully integrated to the Arduino IDE.

but after all, my 2 threads in my fibonacci example appear to run simultaneously, even if 1 thread provides no sleeps in between. Just main loop is blocked, not the 2nd thread. Just the main loop thing seems to be defective FTM (not tried 4 or 5 threads w/o delays yet though)

[atanisoft]

so IIUC, the preemptive scheduler is broken or defective or incomplete for Arduino? Then the Arduino implementation is faulty and thus has to be changed to the full IDF thing. preemptive MT is crucial for std::thread, either which extra libs it will demand, it has to be fully integrated to the Arduino IDE.

arduino-esp32 does not provide ANY part of the scheduler, pthread, std::thread, etc. All of that is coming from ESP-IDF. Feel free to open a bug against the ESP-IDF project for their consideration at implementing what you feel is missing there.

[dsyleixa]

so what has to be changed in ESP-IDF exactly to make it run also in arduino-esp32?

[atanisoft]

so what has to be changed in ESP-IDF exactly to make it run also in arduino-esp32?

That would be up to the ESP-IDF devs, you can take your expectations up with them.

[dsyleixa]

ok, thanks, I assumed that all that about ESP32 is just 1 common project.

[dsptech]

Hi @atanisoft,

The FreeRTOS task scheduler is not a preemptive scheduler, it is a co-operative scheduler.

Are you sure about that ? (See: https://www.freertos.org/RTOS.html)

As I remember (long time ago), the precompiled SDK has the freeRTOS component that work using 1ms timeslice (1Khz clock) and the task switching that occur every clock tick (with context saving and so on....), so, it is preemptive. Of course, you can also use yield and delay functions to switch task cooperatively, but the task switch occur anyway at every interrupt tick (and the control could return to the task itself if there are not same or higher priority tasks that need cpu).

For @dsyleixa, if you need a preemptive solution against the main loop, you need to create the additive task with priority "1", the same used by the Arduino framework. On another side, the Arduino FW is pinned to the CPU core n.1 (APP_CPU) and cannot span to all cores, so, your created task can supersede Arduino FW even if the priority is the same (there are particular conditions that allow that). It's better to create your task pinned to the core 1 too.

I don't know how do it via std::thread, but you could use xTaskCreatePinnedToCore function instead (see ESP-IDF doc).

Regards.

[atanisoft]

As I remember (long time ago), the precompiled SDK has the freeRTOS component that work using 1ms timeslice (1Khz clock) and the task switching that occur every clock tick (with context saving and so on....), so, it is preemptive.

You could be right, but it doesn't get a chance to switch tasks if there is no pause point for the scheduler task itself to run. It very well could be a bug in the implementation for multi-core.

[dsptech]

Hi, the scheduler is attached to an hardware timer interrupt. The interrupt shot occur anyway, regardless the current task job, and interrupt handler perform the forced context saving of the actually active task. The interrupt handler also set the stack pointer to point the stack of a another waiting task and, on the exit from the interrupt handler, the context of the waiting task will be automatically restored causing the resume of that task. This is the task switching as performed by freeRTOS.

.... This is also the reason because the handler of an higher priority interrupt (interrupt priority, not task priority) cannot interact with the scheduler, even using xxxxFromIsr APIs. Regards.

[dsyleixa]

igrr suggested the following - unfortunately it does not compile yet for me, but I'm hoping to get that fixed perhaps, too:

esp_pthread_cfg_t cfg;
  if (esp_pthread_get_cfg(&cfg) != ESP_OK) {
      cfg = esp_pthread_get_default_config();
  }
  cfg.prio=1;
  if (esp_pthread_set_cfg(&cfg) != ESP_OK) {
      printf("esp_pthread_set_cfg failed\n");
      abort();
  };

exit status 1 'esp_pthread_get_default_config' was not declared in this scope https://github.com/espressif/esp-idf/issues/3544#issuecomment-496157019

[dsyleixa]

perhaps the esp32-IDF version used by https://github.com/espressif/arduino-esp32 needs to be updated to a later version? https://github.com/espressif/esp-idf/issues/3544#issuecomment-496913029

[atanisoft]

perhaps the esp32-IDF version used by https://github.com/espressif/arduino-esp32 needs to be updated to a later version? espressif/esp-idf#3544 (comment)

It is intentionally on the v3.2 branch right now as that is the last "stable" release of ESP-IDF. Until there is a new "stable" it will remain on v3.2.

[dsyleixa]

ok, but std::thread prios cannot arbitrarily "stably" be initialized on 3.2 :-/

[atanisoft]

ok, but std::thread prios cannot arbitrarily "stably" be initialized on 3.2 :-/

It can be but you can't get the default config in the manner that was proposed in your referenced issue. Instead you would need to initialize the structure fully in your code and call the set method prior to creating the std::thread instance.

[dsyleixa]

yes, that is still crucial as there is no documentation/example code yet available abot how to do that exactly :-/

[dsyleixa]

@atanisoft : could you please tell me how to solve that?

[atanisoft]

@atanisoft : could you please tell me how to solve that?

This is 100% untested but should work:

#include <esp_pthread.h>

std::thread *counter_loop_thread;

void setup() {
  esp_pthread_cfg_t cfg;
  esp_pthread_get_cfg(&cfg);
  cfg.prio = 1;
  esp_pthread_set_cfg(&cfg);
  counter_loop_thread = new std::thread(counter_loop);
}

This is based on previous examples in this issue, this should give you a start towards a complete example.

[dsyleixa]

unfortunately not:

blinker_loop (HIGH) counter: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: loopTask
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d63a3 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d63a3:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea37e:0x3ffb89e0 0x400ea379:0x3ffb8a00 0x400ea379:0x3ffb8a20 0x400ea379:0x3ffb8a40 0x400ea379:0x3ffb8a60 0x400ea379:0x3ffb8a80 0x400ea379:0x3ffb8aa0 0x400ea379:0x3ffb8ac0 0x400ea379:0x3ffb8ae0 0x400ea379:0x3ffb8b00 0x400ea379:0x3ffb8b20 0x400ea379:0x3ffb8b40 0x400ea379:0x3ffb8b60 0x400d0e0f:0x3ffb8b80 0x400ea365:0x3ffb8bc0 0x400e6ced:0x3ffb8be0 0x400e5db8:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
blinker_loop (HIGH) counter: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: loopTask
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d63a3 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d63a3:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea37e:0x3ffb89e0 0x400ea379:0x3ffb8a00 0x400ea379:0x3ffb8a20 0x400ea379:0x3ffb8a40 0x400ea379:0x3ffb8a60 0x400ea379:0x3ffb8a80 0x400ea379:0x3ffb8aa0 0x400ea379:0x3ffb8ac0 0x400ea379:0x3ffb8ae0 0x400ea379:0x3ffb8b00 0x400ea379:0x3ffb8b20 0x400ea379:0x3ffb8b40 0x400ea379:0x3ffb8b60 0x400d0e0f:0x3ffb8b80 0x400ea365:0x3ffb8bc0 0x400e6ced:0x3ffb8be0 0x400e5db8:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
blinker_loop (HIGH) counter: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: loopTask
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d63a3 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d63a3:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea37e:0x3ffb89e0 0x400ea379:0x3ffb8a00 0x400ea379:0x3ffb8a20 0x400ea379:0x3ffb8a40 0x400ea379:0x3ffb8a60 0x400ea379:0x3ffb8a80 0x400ea379:0x3ffb8aa0 0x400ea379:0x3ffb8ac0 0x400ea379:0x3ffb8ae0 0x400ea379:0x3ffb8b00 0x400ea379:0x3ffb8b20 0x400ea379:0x3ffb8b40 0x400ea379:0x3ffb8b60 0x400d0e0f:0x3ffb8b80 0x400ea365:0x3ffb8bc0 0x400e6ced:0x3ffb8be0 0x400e5db8:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
blinker_loop (HIGH) counter: 0
Fibbonacci of 30=832040

[atanisoft]

Actually, that is working EXACTLY as designed. One of your threads (listed as pthread in the output above, which is likely the default name for std::thread) is not resetting the watchdog and is not delaying/yielding to other tasks so the WDT kicks in and resets.

Either you add a small delay (1-2uS) in your std::thread code OR you feed the WDT so it doesn't trigger a restart.

[stickbreaker]

@dsyleixa If your pthread task does not delay() or yield() WDT will expire. It is working as Designed. Follow the recommendation of @atanisoft.

Chuck.

[dsyleixa]

I can't use delays or yields because it may happen that either task may coincidentally stall at any time (!) and then will hang up the whole system (real code for a mobile robot, not this test!). That is why the code MUST work without any delays! That's exactly why a preemptive MT is needed compellingly! (pthread for my Raspi never hangs up if 1 of the Threads is stalling, and that's how a preemptive MT is designed to work!)

[dsyleixa]

I have no idea though how to feed a watchdog, especially in a stalling thread :-/

[stickbreaker]

Bad design. If your design is that unstable, it will never function in the real world. Your foreground task will be interrupted and delayed by higher priority tasks and interrupts. So if you expect to be able to calculate exact duration of execution you are going to have to design and build your own OS/environment. Something without any interrupt support. Arduino-Esp32 exists in an environment, that environment has to co-exist with real hardware, Input signals by definition are generated externally. You can respond, but you cannot control the timing.

Chuck.

[dsyleixa]

it will work if pthread worked reliably, on my Raspi it does, and here pthread works perfectly preemptive that a system hang-up won't happen if just 1 thread is stalling (tested). E.g., an issue might be a stalling bus system (i2c).

[atanisoft]

I have no idea though how to feed a watchdog, especially in a stalling thread :-/

add a call to feedLoopWDT() which will reset the current task's WDT timeout. This is typically used for the loop task but it isn't required to only be used for that task.

it will work if pthread worked reliably, on my Raspi it does, and here pthread works perfectly preemptive

You can not compare a Linux OS vs the ESP32, there is no valid comparison between the two.

[dsyleixa]

where should I add that feedLoopWDT() additionally at which code line(s)?

[dsyleixa]

I tried this:

// std::thread for ESP32, Arduino IDE

// ver 0.0.5 fibonacci

#include <Arduino.h>
#include <thread>
#include <esp_pthread.h>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

int fibbonacci(int n) {
   if(n == 0){
      return 0;
   } else if(n == 1) {
      return 1;
   } else {
      return (fibbonacci(n-1) + fibbonacci(n-2));
   }
}

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    while(true) {
        feedLoopWDT();
        Serial.println((String)"fibonacci_loop counter: "+counter);  
        for(i=30; i<41; i++) {    // limits: test, debug
          Serial.println( (String)"Fibbonacci of "+i+"="+fibbonacci(i));            
        }                
        counter++;      
    }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        feedLoopWDT();
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println((String)"blinker_loop (HIGH) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.println((String)"blinker_loop (LOW) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

std::thread *thread_1;
std::thread *thread_2;

void setup() {
  Serial.begin(115200);
  //debug
  delay(1000);

  esp_pthread_cfg_t cfg;
  esp_pthread_get_cfg(&cfg);
  cfg.prio = 1;
  esp_pthread_set_cfg(&cfg);

  thread_2 = new std::thread(fibonacci_loop);  
  thread_1 = new std::thread(blinker_loop);

  feedLoopWDT();
}

void loop() {
    static uint32_t main_loop_counter = 0;
    feedLoopWDT();
    Serial.println((String)"main loop: " + main_loop_counter);
    delay(5000);
    main_loop_counter++;
}

result:

fibonacci_loop counter: 0
blinker_loop (HIGH) counter: 0
main loop: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12194) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12194) task_wdt:  - IDLE0 (CPU 0)
E (12194) task_wdt: Tasks currently running:
E (12194) task_wdt: CPU 0: pthread
E (12194) task_wdt: CPU 1: IDLE1
E (12194) task_wdt: Aborting.
abort() was called at PC 0x400d6437 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d6437:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea415:0x3ffb8980 0x400ea40d:0x3ffb89a0 0x400ea40d:0x3ffb89c0 0x400ea40d:0x3ffb89e0 0x400ea40d:0x3ffb8a00 0x400ea40d:0x3ffb8a20 0x400ea40d:0x3ffb8a40 0x400ea40d:0x3ffb8a60 0x400ea40d:0x3ffb8a80 0x400ea40d:0x3ffb8aa0 0x400ea40d:0x3ffb8ac0 0x400ea40d:0x3ffb8ae0 0x400ea40d:0x3ffb8b00 0x400ea40d:0x3ffb8b20 0x400ea40d:0x3ffb8b40 0x400ea40d:0x3ffb8b60 0x400d0e5d:0x3ffb8b80 0x400ea3f9:0x3ffb8bc0 0x400e6d81:0x3ffb8be0 0x400e5e4c:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
fibonacci_loop counter: 0
blinker_loop (HIGH) counter: 0
main loop: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: IDLE1
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d6437 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d6437:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea415:0x3ffb8980 0x400ea40d:0x3ffb89a0 0x400ea40d:0x3ffb89c0 0x400ea40d:0x3ffb89e0 0x400ea40d:0x3ffb8a00 0x400ea40d:0x3ffb8a20 0x400ea40d:0x3ffb8a40 0x400ea40d:0x3ffb8a60 0x400ea40d:0x3ffb8a80 0x400ea40d:0x3ffb8aa0 0x400ea40d:0x3ffb8ac0 0x400ea40d:0x3ffb8ae0 0x400ea40d:0x3ffb8b00 0x400ea40d:0x3ffb8b20 0x400ea40d:0x3ffb8b40 0x400ea40d:0x3ffb8b60 0x400d0e5d:0x3ffb8b80 0x400ea3f9:0x3ffb8bc0 0x400e6d81:0x3ffb8be0 0x400e5e4c:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
fibonacci_loop counter: 0
blinker_loop (HIGH) counter: 0
main loop: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: IDLE1
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d6437 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d6437:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea415:0x3ffb8980 0x400ea40d:0x3ffb89a0 0x400ea40d:0x3ffb89c0 0x400ea40d:0x3ffb89e0 0x400ea40d:0x3ffb8a00 0x400ea40d:0x3ffb8a20 0x400ea40d:0x3ffb8a40 0x400ea40d:0x3ffb8a60 0x400ea40d:0x3ffb8a80 0x400ea40d:0x3ffb8aa0 0x400ea40d:0x3ffb8ac0 0x400ea40d:0x3ffb8ae0 0x400ea40d:0x3ffb8b00 0x400ea40d:0x3ffb8b20 0x400ea40d:0x3ffb8b40 0x400ea40d:0x3ffb8b60 0x400d0e5d:0x3ffb8b80 0x400ea3f9:0x3ffb8bc0 0x400e6d81:0x3ffb8be0 0x400e5e4c:0x3ffb8c10 0x40087c9d:0x3ffb8c30

Rebooting...

[stickbreaker]

Just out of curiosity, how long does your recursive fib() take?

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    while(true) {
        Serial.printf("(%dms) fibonacci_loop counter: %d\n",millis(),counter);  
        for(i=30; i<41; i++) {    // limits: test, debug
        feedLoopWDT();
          Serial.printf("(%dms) Fibbonacci of %d =%d\n",i,fibbonacci(i));            
        }                
        counter++;      
    }
}

Chuck.

[dsyleixa]

fibonacci(40): about 15 sec all fibonacci (30...40): about 45 sec

[dsyleixa]

next try: outsourcing the main loop to an extra thread_3: thread_3 = new std::thread(former_main_loop);

now again the "(former_)main_loop does not work again, even as it now runs at the same prio as the other threads! and THAT can actually not be reasonable!

// std::thread for ESP32, Arduino IDE

// ver 0.0.5 fibonacci

#include <Arduino.h>
#include <thread>
#include <esp_pthread.h>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

int fibbonacci(int n) {
   if(n == 0){
      return 0;
   } else if(n == 1) {
      return 1;
   } else {
      return (fibbonacci(n-1) + fibbonacci(n-2));
   }
}

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    while(true) {
        Serial.println((String)"fibonacci_loop counter: "+counter);  
        for(i=30; i<41; i++) {    // limits: test, debug
          Serial.println( (String)"Fibbonacci of "+i+"="+fibbonacci(i));            
        }                
        counter++;      
    }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println((String)"blinker_loop (HIGH) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.println((String)"blinker_loop (LOW) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

void former_main_loop() {
    static uint32_t main_loop_counter = 0;
    Serial.println((String)"main loop: " + main_loop_counter);
    delay(5000);
    main_loop_counter++;
}

std::thread *thread_1;
std::thread *thread_2;
std::thread *thread_3;

void setup() {
  Serial.begin(115200);
  //debug
  delay(1000);

  thread_1 = new std::thread(blinker_loop);
  thread_2 = new std::thread(fibonacci_loop);  
  thread_3 = new std::thread(former_main_loop);

}

void loop() {}

blinker_loop (HIGH) counter: 0
fibonacci_loop counter: 0
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
blinker_loop (LOW) counter: 2
blinker_loop (HIGH) counter: 3
Fibbonacci of 36=14930352
blinker_loop (LOW) counter: 3
blinker_loop (HIGH) counter: 4
blinker_loop (LOW) counter: 4
blinker_loop (HIGH) counter: 5
Fibbonacci of 37=24157817
blinker_loop (LOW) counter: 5
blinker_loop (HIGH) counter: 6
blinker_loop (LOW) counter: 6
blinker_loop (HIGH) counter: 7
blinker_loop (LOW) counter: 7
blinker_loop (HIGH) counter: 8
blinker_loop (LOW) counter: 8
Fibbonacci of 38=39088169
blinker_loop (HIGH) counter: 9
blinker_loop (LOW) counter: 9
blinker_loop (HIGH) counter: 10
blinker_loop (LOW) counter: 10
blinker_loop (HIGH) counter: 11
blinker_loop (LOW) counter: 11
blinker_loop (HIGH) counter: 12
blinker_loop (LOW) counter: 12
blinker_loop (HIGH) counter: 13
blinker_loop (LOW) counter: 13
blinker_loop (HIGH) counter: 14
blinker_loop (LOW) counter: 14
Fibbonacci of 39=63245986
blinker_loop (HIGH) counter: 15
blinker_loop (LOW) counter: 15
blinker_loop (HIGH) counter: 16
blinker_loop (LOW) counter: 16
blinker_loop (HIGH) counter: 17
blinker_loop (LOW) counter: 17
blinker_loop (HIGH) counter: 18
blinker_loop (LOW) counter: 18
blinker_loop (HIGH) counter: 19
blinker_loop (LOW) counter: 19
blinker_loop (HIGH) counter: 20
blinker_loop (LOW) counter: 20
blinker_loop (HIGH) counter: 21
blinker_loop (LOW) counter: 21
blinker_loop (HIGH) counter: 22
blinker_loop (LOW) counter: 22
blinker_loop (HIGH) counter: 23
blinker_loop (LOW) counter: 23
Fibbonacci of 40=102334155
fibonacci_loop counter: 1
Fibbonacci of 30=832040
Fibbonacci of 31=1346269
blinker_loop (HIGH) counter: 24
Fibbonacci of 32=2178309
Fibbonacci of 33=3524578
blinker_loop (LOW) counter: 24
Fibbonacci of 34=5702887
blinker_loop (HIGH) counter: 25
blinker_loop (LOW) counter: 25
Fibbonacci of 35=9227465
blinker_loop (HIGH) counter: 26
blinker_loop (LOW) counter: 26
Fibbonacci of 36=14930352
blinker_loop (HIGH) counter: 27
blinker_loop (LOW) counter: 27
blinker_loop (HIGH) counter: 28
blinker_loop (LOW) counter: 28
blinker_loop (HIGH) counter: 29
Fibbonacci of 37=24157817
blinker_loop (LOW) counter: 29
blinker_loop (HIGH) counter: 30
blinker_loop (LOW) counter: 30
blinker_loop (HIGH) counter: 31
blinker_loop (LOW) counter: 31
blinker_loop (HIGH) counter: 32
blinker_loop (LOW) counter: 32
Fibbonacci of 38=39088169
blinker_loop (HIGH) counter: 33
blinker_loop (LOW) counter: 33
blinker_loop (HIGH) counter: 34
blinker_loop (LOW) counter: 34
blinker_loop (HIGH) counter: 35
blinker_loop (LOW) counter: 35
blinker_loop (HIGH) counter: 36
blinker_loop (LOW) counter: 36
blinker_loop (HIGH) counter: 37
blinker_loop (LOW) counter: 37
blinker_loop (HIGH) counter: 38
Fibbonacci of 39=63245986
blinker_loop (LOW) counter: 38
blinker_loop (HIGH) counter: 39
blinker_loop (LOW) counter: 39
blinker_loop (HIGH) counter: 40
blinker_loop (LOW) counter: 40
blinker_loop (HIGH) counter: 41
blinker_loop (LOW) counter: 41
blinker_loop (HIGH) counter: 42
blinker_loop (LOW) counter: 42
blinker_loop (HIGH) counter: 43
blinker_loop (LOW) counter: 43
blinker_loop (HIGH) counter: 44
blinker_loop (LOW) counter: 44
blinker_loop (HIGH) counter: 45
blinker_loop (LOW) counter: 45
blinker_loop (HIGH) counter: 46
blinker_loop (LOW) counter: 46
blinker_loop (HIGH) counter: 47
Fibbonacci of 40=102334155
fibonacci_loop counter: 2
Fibbonacci of 30=832040
Fibbonacci of 31=1346269

[stickbreaker]

WDT period is less than that, so you have to reset inside loop

[dsyleixa]

I can't reset inside the loop because in "real life" I do not not know where and/or when it may stall, or even how long. The preemptive roundrobin scheduler is expected to work reliably even then, that is what preemptivity finally is about 8)

[atanisoft]

The preemptive roundrobin scheduler is expected to work reliably even then, that is what preemptivity finally is about 8)

What part of the std::thread specification states that it is a preemptive scheduler that manages the tasks, I can't find anything that states that it is required to be that way.

There is also no guarantee that the pthread implementation will always use a preemptive kernel.

[stickbreaker]

The WDT period is configured when me-no-dev generates the IDF libraries, I don't know the current value, I think it is about 8 seconds. So, any task needs to complete in less than that duration. if fib(40) takes 15 seconds, then inside of fib() there needs to be a WDT reset call. If you are worried about a specific task locking up, you can add your specific task to the WDT list, and require it to clear it's own WDT. esp_task_wdt_add() this applies to a FreeRTOS task, It looks like the STD::THREAD lives somewhere else.

Chuck.

[dsyleixa]

imagine a i2c device don't answer to a read or write command because of any unknown reason, and the device lib cuts the connection (intermediately or finally), not proceeding with anything afterwards. Or perhaps another unforeseen issue might have caused that thread to stall intermediately. As the main task now also stalls, I cannot even evaluate heartbeats any more to detach or join that thread and restart the i2c bus and the thread anew (assuming it was possible).

However, it would be absolutely no problem if the scheduler relentlessly switched e.g. every 1 ms to the next time slice for the next process/thread, independent of everything else.

If that issue is crucial and ineluctable and sort of an immutable basic (mal-)function to a ESP32, then I have to stay with the Raspi unfortunately.

[stickbreaker]

Sounds like a plan, You want/need a complex system that supports a preemptive multitasking, so by spending $35+$10(sdcard) an you will have a much more capably system than a $6 esp32.

Chuck.

[dsyleixa]

yes, I have that already, but I was thinking of downgrading because of the simpler Arduino API, also for WiFi and Webserver vs. C/C++ + makefile/cmake + wiringPi/pigpio and pthread, also for a planned community project. A SD card is always at my Arduinos, and my Adafruit Feather ESP32 costed 24 EUR.

[Xx220xX]

When we don't know something, we search and study. If you don't know c ++ I recommend you get a book (it's "free") and study because arduino is programmed in object oriented c ++. I have a library made for this kind of work of yours. https://github.com/Xx220xX/ThreadEsp32 To install will include include zip library in arduino Ide (if it is very difficult to do this give up programming) then go to Examples and have two examples ps :: is exclusively for Esp32

[Masterlong-Dragon]

Actually, the FreeRTOS function vTaskPrioritySet() is available in the std::thread here.

// std::thread for ESP32, Arduino IDE

// ver 0.0.5 fibonacci

#include <Arduino.h>
#include <thread>
#include <freertos/task.h>

#ifndef LED_BUILTIN
#define LED_BUILTIN 13
#endif

const auto one_sec = std::chrono::seconds
{
    1
};

int fibbonacci(int n) {
   if(n == 0){
      return 0;
   } else if(n == 1) {
      return 1;
   } else {
      return (fibbonacci(n-1) + fibbonacci(n-2));
   }
}

void blinker_loop() {
    thread_local uint32_t counter = 0;
    Serial.println((String)"blinker_loop Current priority :" + uxTaskPriorityGet(NULL));
    vTaskPrioritySet(NULL,1);//set Priority
    Serial.println((String)"blinker_loop Current priority :" + uxTaskPriorityGet(NULL));
    while(true) {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println((String)"blinker_loop (HIGH) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        digitalWrite(LED_BUILTIN, LOW);
        Serial.println((String)"blinker_loop (LOW) counter: "+ counter);
        std::this_thread::sleep_for(one_sec);

        counter++;
    }
}

void fibonacci_loop() {
    thread_local uint32_t counter = 0, i=0;
    Serial.println((String)"fibonacci_loop Current priority :" + uxTaskPriorityGet(NULL));
    vTaskPrioritySet(NULL,1);//set Priority
    Serial.println((String)"fibonacci_loop Current priority :" + uxTaskPriorityGet(NULL));
    while(true) {
        for(i=30; i<41; i++) {    // limits: test, debug
          Serial.println( (String)"Fibbonacci of "+i+"="+fibbonacci(i));            
        }        
        Serial.println((String)"fibonacci_loop counter: "+counter);  
        counter++;      
    }
}

std::thread *thread_1;
std::thread *thread_2;

void setup() {
  Serial.begin(115200);
  delay(1000);
  thread_1 = new std::thread(blinker_loop);
  thread_2 = new std::thread(fibonacci_loop);
}

void loop() {
    delay(500);
    static uint32_t main_loop_counter = 0;
    Serial.println((String)"main loop: " + main_loop_counter);
    delay(5000);
    main_loop_counter++;
}

result：

blinker_loop Current priority :5
fibonacci_loop Current priority :5
blinker_loop Current priority :1
fibonacci_loop Current priority :1blinker_loop (HIGH) counter: 0

Fibbonacci of 30=832040
Fibbonacci of 31=1346269
main loop: 0
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
blinker_loop (LOW) counter: 2
main loop: 1
blinker_loop (HIGH) counter: 3
Fibbonacci of 36=14930352
blinker_loop (LOW) counter: 3
blinker_loop (HIGH) counter: 4
blinker_loop (LOW) counter: 4
blinker_loop (HIGH) counter: 5
Fibbonacci of 37=24157817
blinker_loop (LOW) counter: 5
main loop: 2
blinker_loop (HIGH) counter: 6
blinker_loop (LOW) counter: 6

[dsyleixa]

thank you, this function you mentioned vTaskPrioritySet(NULL,1);//set Priority is new to me, I didn't find that so far although asked for sth like that here in this repo. Thanks for your contribution!

[dsyleixa]

update

I tried that code but I always get a reboot after Fibonacci 35 because of watchdog issues:

Fibbonacci of 30=832040
Fibbonacci of 31=1346269
main loop: 0
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: IDLE1
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d6577 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d6577:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea4f5:0x3ffb93f0 0x400ea4f5:0x3ffb9410 0x400ea4f5:0x3ffb9430 0x400ea4f5:0x3ffb9450 0x400ea4f5:0x3ffb9470 0x400ea4f5:0x3ffb9490 0x400ea4f5:0x3ffb94b0 0x400ea4f5:0x3ffb94d0 0x400ea4f5:0x3ffb94f0 0x400ea4f5:0x3ffb9510 0x400ea4f5:0x3ffb9530 0x400ea4f5:0x3ffb9550 0x400ea4f5:0x3ffb9570 0x400ea4f5:0x3ffb9590 0x400ea4f5:0x3ffb95b0 0x400d0ec3:0x3ffb95d0 0x400ea4e1:0x3ffb9620 0x400e6e69:0x3ffb9640 0x400e5f34:0x3ffb9670 0x40087c9d:0x3ffb9690

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
blinker_loop Current priority :5
blinker_loop Current priority :1
fibonacci_loop Current priority :5
blinker_loop (HIGH) counter: 0fibonacci_loop Current priority :1

Fibbonacci of 30=832040
Fibbonacci of 31=1346269
main loop: 0
Fibbonacci of 32=2178309
blinker_loop (LOW) counter: 0
Fibbonacci of 33=3524578
blinker_loop (HIGH) counter: 1
Fibbonacci of 34=5702887
blinker_loop (LOW) counter: 1
blinker_loop (HIGH) counter: 2
Fibbonacci of 35=9227465
E (12195) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (12195) task_wdt:  - IDLE0 (CPU 0)
E (12195) task_wdt: Tasks currently running:
E (12195) task_wdt: CPU 0: pthread
E (12195) task_wdt: CPU 1: IDLE1
E (12195) task_wdt: Aborting.
abort() was called at PC 0x400d6577 on core 0

Backtrace: 0x40089150:0x3ffbe160 0x4008937d:0x3ffbe180 0x400d6577:0x3ffbe1a0 0x40081671:0x3ffbe1c0 0x400ea4f5:0x3ffb93f0 0x400ea4f5:0x3ffb9410 0x400ea4f5:0x3ffb9430 0x400ea4f5:0x3ffb9450 0x400ea4f5:0x3ffb9470 0x400ea4f5:0x3ffb9490 0x400ea4f5:0x3ffb94b0 0x400ea4f5:0x3ffb94d0 0x400ea4f5:0x3ffb94f0 0x400ea4f5:0x3ffb9510 0x400ea4f5:0x3ffb9530 0x400ea4f5:0x3ffb9550 0x400ea4f5:0x3ffb9570 0x400ea4f5:0x3ffb9590 0x400ea4f5:0x3ffb95b0 0x400d0ec3:0x3ffb95d0 0x400ea4e1:0x3ffb9620 0x400e6e69:0x3ffb9640 0x400e5f34:0x3ffb9670 0x40087c9d:0x3ffb9690

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:8424
ho 0 tail 12 room 4
load:0x40080400,len:5868
entry 0x4008069c
blinker_loop Current priority :5
blinker_loop Current priority :1
fibonacci_loop Current priority :5
blinker_loop (HIGH) counter: 0fibonacci_loop Current priority :1

Fibbonacci of 30=832040

[Masterlong-Dragon]

Hmm, as stickbreaker said, there needs to be a WDT reset call. std::thread here is implemented on top of FreeRTOS in ESP-IDF. Therefore, you can simply increase the WDT timeout value in setup() like this: #include <esp_task_wdt.h>

#define TWDT_TIMEOUT_S 20
  //Initialize or reinitialize TWDT
  esp_task_wdt_init(TWDT_TIMEOUT_S, false);
  //Subscribe Idle Tasks to TWDT if they were not subscribed at startup
#ifndef CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0
  esp_task_wdt_add(xTaskGetIdleTaskHandleForCPU(0));
#endif
#ifndef CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1
  esp_task_wdt_add(xTaskGetIdleTaskHandleForCPU(1));

However, it's not a solution at all... If you use esp_task_wdt_init(TWDT_TIMEOUT_S, true);, you will got warnings but it won't lead to rebooting. To avoid that WDT got triggered, you can disable WDT in this way, which works for me:

  //Check which core are Idle Tasks in and disable the core's WDT
#ifndef CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0
  disableCore0WDT();
#endif
#ifndef CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1
  disableCore1WDT();
#endif

But I'm not sure whether there will be any errors caused by blocked tasks.

[me21]

This is non-issue, all that's needed is to set task priority lower than loop task, if I'm not mistaken. Then loop task will feed the watchdog.

And of course, some time should be invested into preventing your tasks from stalling. Stalling a task because of faulty i2c device is definitely a bug.

[dsyleixa]

@Masterlong-Dragon how did YOU manage the program to work? I took you code, your serial output from https://github.com/espressif/arduino-esp32/issues/2814#issuecomment-515952643 seemed to be ok, but for me it does not work (Arduino IDE: 1.8.9, board manager: latest ESP32 core 1.0.2)

[dsyleixa]

@me21: Thank you, I have to agree, when setting the thread to prio 0 then it works!

JTM: IMO it would be better to set the main loop to a higher prio by default (e.g., 3) to have a wider bandwidth fo thread prios to cooperate!

[stale[bot]]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.