Long-running callbacks

[lackdaz]

Fantastic library btw! I've been using it for a while to get rid of my watchdog timers but I've encountered a problem with using yield(). I currently running some code on a atmega2560.

Say I've got two tasks, t1 running a stepper pulse drive every 200 uS and t2 running long-running callback (>200 uS, say 800uS). can Task.yield(&t1) be used in this way to defer to the scheduler to run t1 in t2 mid-processing time? Right now I seem to be getting a lot of jitter on my stepper motor because of t2's long processing time. Any suggestions on how to solve this will be welcome!

Right now, I'm able to break up t2 into smaller methods (it's invoking OO class methods), e.g.

// long-running callback
void t2_callback(){
   do_this_first();
   // hopefully I can yield() here
   then_call_this();
}

a way that I've identified to solve this is to simply break up the long running t2_callback into two separate tasks -- an inelegant but probably my most likely solution.

Or would this be something StatusRequest can handle?

[arkhipenko]

Thank you. Could you please send me the code for t1 and t2? I need to see if t2 could be split somehow? An instance of how I addressed the long "running" tasks could be found here: https://github.com/arkhipenko/DFRobotDFPlayerMini#updates-by-anatoli-arkhipenko

There was a hardware mandated delay which I had to split the callback routine over. Worked fine.

[arkhipenko]

And no yield() unfortunately is not the way to switch context. This is a purely cooperative library, so the task has to return. TS's yield() is just a return with a switch of callback routine pointer.

[arkhipenko]

Something like this might work:

// long-running callback
void t2_callback1(){
   do_this_first();
  yield(&t2_callback2);
}

// long-running callback
void t2_callback2(){
   then_call_this();
   t2.setCallback(&t2_callback1);
}

[lackdaz]

And no yield() unfortunately is not the way to switch context. This is a purely cooperative library, so the task has to return. TS's yield() is just a return with a switch of callback routine pointer.

Ok, got it. Let me digest what you just linked. t2 can be split into two tasks but if it has got to be way then it shall be! I would love to send you the code but its a giant monolith right now in the middle of a refactor.

It's actually a routine running 4 thermistor readings and pid controlling 4 separate heaters.

There is also going to be a OLED display sometime later so I've got to be very careful about the time slices since its also handling stepper output at the moment

[arkhipenko]

What microcontroller are you running on? For the tasks like this I find ESP32 with a combination of preemptive FreeRTOS for critical functions (steppers) and TaskScheduler for the rest could be an interesting solution, but you have to be very careful with RTOS task definitions (not to starve TS running task)

[arkhipenko]

Or use new Teensy 4 - with 600Mhz you should be able to do everything and some. I used Teensy 3.5 for my predator costume and was able to read 2 accelerometers, control 2 servos, a bunch of LEDs and communicate over BT without breaking a sweat. 200Mhz

[lackdaz]

I needed 4 x pwm pins for this so the ESP32 was out with its x2

I contemplated breaking out the Teensy 4.0, but I haven't checked if it has enough IO. Currently I'm using up nearly 20 IO pins on this little robotics project of mine.

I'm actually using 2 cross-talking arduinos to handle all the operations (x1 stepper, x4 pid heaters, x2 linebreak sensors, x3 buttons, x1 solenoid valve, OLED) at the moment and that works fine. But this refactor was to see if a single atmega2560 could handle everything. *crossing fingers*

[lackdaz]

I'll mark this as closed. Thank you again for your time, this library is phenomenal and just reading your code pushed the boundaries (and limits) of what I could do with my microcontrollers

[arkhipenko]

The ESP32 LED PWM controller has 16 independent channels that can be configured to generate PWM signals with different properties. All pins that can act as outputs can be used as PWM pins (GPIOs 34 to 39 can't generate PWM).

[lackdaz]

Wait so every pin on the ESP32 can act as a PWM? (except the strict input ones?) I was reading off the motor PWM with its 2 x 6 channels.

https://riot-os.org/api/group__boards__esp32__wemos-lolin-d32-pro.html

[arkhipenko]

I am definitely playing a PWM sound on pin 32:

const int SOUND_PIN = 32;
const int CHANNEL = 0;
volatile float toneFrequency = 400;
  // setup pwm channel on sound pin
  ledcSetup(CHANNEL, toneFrequency, 10);
  ledcAttachPin(SOUND_PIN, CHANNEL);
    ledcWriteTone(CHANNEL, toneFrequency);
delay(1000);
    ledcWrite(CHANNEL, 0);

[arkhipenko]

https://randomnerdtutorials.com/esp32-dc-motor-l298n-motor-driver-control-speed-direction/

[arkhipenko]

They use ledc as well

[lackdaz]

Goodness me, I've been blindsided for some time now. Keep in touch, I'll post you any updates on my integration.

[arkhipenko]

Also this one is awesome! https://www.amazon.com/Adafruit-16-Channel-12-bit-Servo-Driver/dp/B00KLDPYXO/

[lackdaz]

Its out of stock on the adafruit website but I'll look into it for sure.

@arkhipenko One more question, if I want to benchmark all my running tasks is there currently a facility to do that? I'm looking to print task execution interval of each running task as defined in your time critical wiki. From what I catch,t.getOverrun yields the leftover time left after factoring in task execution time.

Another question regarding cooperative multitasking is, if a task overruns, say t1, and eats into the scheduled time taken by other tasks (say t2, t3). Does the now delayed scheduled time now shift the time horizon for the remaining tasks?

so t1 (scheduled to run every 2 seconds, runs 0.5 seconds long) t2 (scheduled to run every 1 second, super quick)

event horizon: t-0: t1 runs t-0.5: t1 ends. t2 starts/ends (delayed by 0.5s) t-1.0: Does t2 run here? t-1.5: Or here instead?

[arkhipenko]

if I want to benchmark all my running tasks is there currently a facility to do that?

Very limited. The scheduler does not persist much of any runtime tasks scheduling information. Only the latest one. What I usually do is I print the current millis() at the entry point of every task. I see how you would be interested to know the same right after the task returns. Currently, you just have to print millis() right before returning (all returns - not elegant I agree). I might think of some facilities.
It will have to be yet another compile-time define (something like _TASK_PROFILING) since it has to be in the main scheduling loop and I am keeping that loop as clean and efficient as I can.

long getStartDelay(); will tell your task how much later it started compared to the scheduled time. long getOverrun(); will tell your task if it is in a "catch up" mode. Catch up mode is when the next scheduled start time of the task is already in the past. Let's say a task is scheduled to run every 100 ms. First run at 0 ms - ok, the second run at 125 ms- still ok (late, but not overrun), third run at 325 ms - overrun 25 ms (after invocation at 125 ms, next scheduled run should be at 200 ms. anything between 200 ms and 300 ms will still be "within" the interval. Anything over 300 ms will be an "overrun").

Does the now delayed scheduled time now shift the time horizon for the remaining tasks?

Late execution of a task definitely affects the start time of the subsequent tasks. But TS is built with priority for schedule, not interval. Which means that TS will try its best to bring tasks back on the original schedule, even if it means running task immediately. So a task scheduled to run every 100 ms will be scheduled to run at 0, 100, 200, 300... or as close to that schedule as possible. If it finishes at 134 ms, next invocation will be still at 200 ms. If the first run finishes at 234 ms, the next invocation will be immediate. If interval is a priority, you need to decide from which point in the task callback you want to count the interval (start/middle/return some other) and call task.delay(interval) at that point of the task. that would push the next itaration. For instance, if you want to read an accelerometer at 100hz and that frequency is important, you call delay(10) right before the call to get data from the accelerometer.

makes sense?

[lackdaz]

Absolutely, I pored through nearly every detail of your library and I think I now finally understand most of it. I just couldnt really tease out why we needed to know the overrun vs delay initially. But it seems now that has become clear as well.

Yeah timesharing was never a big concern for me until I started working in the microsecond range that your library has allowed. So I'm starting to become more performance minded when managing the timeshare on the scheduler. So hence the watchdog timers.

Update: I've split the tasks up and it's been working great so far on my mega so far. Something interesting though, when I switched to an UNO, a serial.print comment in two offending tasks caused the code to not run at all. But when removed it ran smoothly. I'll try to submit to you something reproduceable and this is subject to only the UNO.

On Thu, 3 Oct 2019, 8:27 PM Anatoli Arkhipenko, notifications@github.com wrote:

if I want to benchmark all my running tasks is there currently a facility to do that? Very limited. The scheduler does not persist much of any runtime tasks scheduling information. Only the latest one. What I usually do is I print the current millis() at the entry point of every task. I see how you would be interested to know the same right after the task returns. Currently, you just have to print millis() right before returning (all returns - not elegant I agree). I might think of some facilities. It will have to be yet another compile-time define (something like _TASK_PROFILING) since it has to be in the main scheduling loop and I am keeping that loop as clean and efficient as I can.

long getStartDelay(); will tell your task how much later it started compared to the scheduled time. long getOverrun(); will tell your task if it is in a "catch up" mode. Catch up mode is when the next scheduled start time of the task is already in the past. Let's say a task is scheduled to run every 100 ms. First run at 0 ms - ok, the second run at 125 ms- still ok (late, but not overrun), third run at 325 ms - overrun 25 ms (after invocation at 125 ms, next scheduled run should be at 200 ms. anything between 200 ms and 300 ms will still be "within" the interval. Anything over 300 ms will be an "overrun").

Does the now delayed scheduled time now shift the time horizon for the remaining tasks? Late execution of a task definitely affects the start time of the subsequent tasks. But TS is built with priority for schedule, not interval. Which means that TS will try it's best to bring tasks back on the original schedule, even if it means running task immediately. So a task scheduled to run every 100 ms will be scheduled to run at 0, 100, 200, 300... or as close to that schedule as possible. If it finishes at 134 ms, next invocation will be still at 200 ms. If interval is a priority, you need to decide from which point in the task callback you want to count the interval (start/middle/return some other) and call delay(interval) at that point of the task. that would push the next itaration. For instance, if you want to read an accelerometer at 100hz and that frequency is important, you call delay(10) right before the call to get data from the accelerometer.

makes sense?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/arkhipenko/TaskScheduler/issues/78?email_source=notifications&email_token=AEOAY6HMQVVQ65T6O5ACUWLQMXQLNA5CNFSM4I4Y7OP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAIA45I#issuecomment-537923189, or mute the thread https://github.com/notifications/unsubscribe-auth/AEOAY6ENAP5RKNL6J2TF3RTQMXQLNANCNFSM4I4Y7OPQ .

[lackdaz]

for tasks that run only once (e.g. baton passing between callbacks), do they automatically get disabled after its last iteration?

Like for instance for a stepper motor, I'm hoping to use onEnable, and onDisable to turn the en pin on/off after a move.

[GitMoDu]

Another question you should ask, is if the OnEnable get fired if your task has auto-start.

[arkhipenko]

for tasks that run only once (e.g. baton passing between callbacks), do they automatically get disabled after its last iteration?

All tasks that have limited number of iterations are disabled after the last iteration. Now, in case of task.yield() the iteration count does not increase, because it is considered to be just a break in a single iteration. So in case of this code:

// long-running callback
void t2_callback1(){
   do_this_first();
  yield(&t2_callback2);
}
// long-running callback
void t2_callback2(){
   then_call_this();
   t2.setCallback(&t2_callback1);
}

let's say task t2 is created with a single iteration count. Then on scheduling pass 1 callback1 will be invoked (given of course all other conditions are met), and on scheduling pass 2 callback2 will be invoked, but the run counter in callback2 will still be 1. If you want run counter in callback2 to be 2, then you need to use task.setCallback() method. (And of course, the number of iterations should be >1). I suggest you experiment a little with those methods, they serve slightly different purposes.

Now, about onEnable and onDisable. You have to be carefull with those because onEnable fires up immediately when the task is enabled. In many cases (in my own development) I forgot about it and stuff was happening way before I needed it. If your task activation is on schedule though, then it is a great way to prepare stuff for task execution (like set pins, turn on power relays, etc. etc.) There is a difference between task being enabled and ready to be invoked though. For instance a statement like this: t1.enableDelayed(200); will fire the OnEnable immediately, and first iteration of a callback only after 200 ms.

Another question you should ask, is if the OnEnable get fired if your task has auto-start.

What do you mean by "auto-start"?

[arkhipenko]

By the way, if you just need to do something different at a first or last iteration of the task, use task.isFirstIteration() or task.isLastIteration() methods. Then you don't have to worry about scheduling. https://github.com/arkhipenko/scarab/blob/master/scarab.ino#L371

Or switch the callbacks at the end of the first iteration, like here: https://github.com/arkhipenko/apis/blob/master/apis.ino#L967

[lackdaz]

Yeah I'm actually starting to notice these fine differences. I actually broke my whole motor control flow into a series of callbacks to fire off one after another using a single task called task_motor_control_flow and swapping the callbacks in and out using yieldOnce(&callback). It runs the entire chain but I haven't got it to fire off again after completing its loop (probably some bugs on my end)

I actually ported the whole project to ESP32 but then realised that some of my sensors are running at 5V so I had to revert back to the mega (didn't want to add a level shifter yet)

I experimented using onEnable and onDisable callbacks to add the watchdog timers in there but it only fires off once. I think it might be a case of yieldOnce not going through the entire life cycle because its interrupted there.

Also... I tried to refactor in some code reusability but it seems like the callbacks cannot be passed any parameters? I wanted to pass in some object pointers... e.g.

void heater_update_callback(Heater *heater) {
  heater -> update();
}

which I had hope to use a template to use my heater objects now that I've broken the long execution callback into its separate objects and creating different tasks for them. I just wanted to confirm that this is the design because I'm still rather new to the OO way to doing things in C++

[lackdaz]

right now my watchdog timer is just two functions like these:

bool wdt_start() {
  wdt_1 = micros();
  return true;
}

void wdt_stop() {
  wdt_2 = micros() - wdt_1;
  Serial.print("execution interval: ");
  Serial.println(wdt_2);
}

and an implementation looks like this:

void heater1_update_callback() {
  #ifdef DEBUG_WDT
    wdt_start();
    heater1 -> update();
    wdt_stop();
  #else
    heater1 -> update();
  #endif

wdt_start did not need to return but its just a legacy of using onEnable with it

So the actual code that is timed is just sandwiched in between, not great - but its the only thing that I've got working so far. And I would definitely want it turned off in production. This code doesn't look very DRY, any suggestions how I might be able to correct this if I'm not able to pass the objects into the function parameters?

[arkhipenko]

I actually ported the whole project to ESP32 but then realised that some of my sensors are running at 5V so I had to revert back to the mega (didn't want to add a level shifter yet)

Some of the ESP32 pins are 5v tolerant (I believe - please do check) so you can easily read from the 5V sensors. Controlling a 5V device usually also works because of a logical "1" working for both 5V and 3.3V levels. Using a level shifter is preferrable of course. I just think the power you get and ability use WiFI and BT is a great asset!

I think it might be a case of yieldOnce not going through the entire life cycle because its interrupted there.

Yes, you got it - better use taks.yield() instead. Basically if you have broken the long-running callback, say, into 3, the sequence should be:

void callback1() {
  ...
  t.yield(&callback2);
}

void callback2() {
  ...
  t.yield(&callback3);
}

void callback3() {
  ...
  t.setCallback(&callback1);
}

This way task will go over all of its iterations. Please do remember that all three callbacks will have the same iteration counter:

Pass task.Iteration callback 1 1 1 2 1 2 3 1 3 4 2 1 5 2 2 6 2 3 7 3 1 and so forth.

it seems like the callbacks cannot be passed any parameters?

No. You can use Task Local Storage Pointer instead or switch to OO way of task inheritance - but latter is a big refactoring work! LTS doc: https://github.com/arkhipenko/TaskScheduler/wiki/API-Task#local-task-storage-methods-1

and here: https://github.com/arkhipenko/TaskScheduler/wiki/Implementation-scenarios-and-ideas#7-using-local-task-storage-pointer

example: https://github.com/arkhipenko/TaskScheduler/tree/master/examples/Scheduler_example08_LTS

OO callbacks example (if you are ready to go that way): https://github.com/arkhipenko/TaskScheduler/tree/master/examples/Scheduler_example21_OO_Callbacks

wdt_start did not need to return but its just a legacy of using onEnable with it

I would arm wdt during the first task iteration, not in the OnEnable.
The reason is OnEnable fires off when you activate the task, not when it's scheduled to run.

If you are using Arduino, then a good example of WDT use is here:https://github.com/arkhipenko/TaskScheduler/tree/master/examples/Scheduler_example07_WDT

It basically uses a special HeartBeat task to reset WDT. Any too-long running callback will block the entire chain and trigger WDT interrupt. You can use the task IDs and control points methods to identify the culprit (i,e,, write the information into EEPROM). https://github.com/arkhipenko/TaskScheduler/wiki/API-Task#task-id-control-points-methods

Hope this helps.

[lackdaz]

@arkhipenko Thank you again for your detailed response. I will have to take some time to digest this.

Some of the ESP32 pins are 5v tolerant (I believe - please do check) so you can easily read from the 5V sensors. Controlling a 5V device usually also works because of a logical "1" working for both 5V and 3.3V levels. Using a level shifter is preferrable of course. I just think the power you get and ability use WiFI and BT is a great asset!

Yes, you got it - better use taks.yield() instead. Basically if you have broken the long-running callback, say, into 3, the sequence should be:

I've done quite a bit of cross platform portability with my code, like reintroducing arduino-style PWM writes, separate object initialisations with pwm channels --- ESP32 requires a lot more handwiring!). But the nail in the coffin was the 3.3v output not being able to fully saturate my mosfet gates leading to an initially very hot mosfet when doing pwm, and it was only with a 5V signal that this was solved. This happened so long ago that I forgot about it - so it was either I changed the mosfets to something beefier or changed the microcontroller.

I initially tried to use yield(&callback) but there were errors going about incorrect parameters so I assumed it was an ESP8266 exclusive function. I did have success using setCallback(&callback) and either restart or enable. I'll try this again to see what the error was and report back again.

I'd definitely need to swing back and experiment with this area once I've completed the code sprint. I think what needs to be codified in the wiki is basically what's the best practice for single running tasks with multiple callbacks e.g. t1.enable() -> t1.callbackA() -> t1.callbackB() -> t1.callbackC() -> t1.disable(), repeat, and so on. I'm guessing the different functions are specifically to preserve the current number of iterations vs restarting. But for single-run use cases, having between yield, enable and restart are afaik the different ways to skin this cat.

I actually saw the WDT example earlier and was very excited about the task ids and control points potentially being able to implement multiple watchdog timers (like js's console.time(timer1)).

I'm completely prepared to go the OO route, and thank you for pointing the way!