Best way to run something every N seconds, precisely

[adrfantini]

Hi! I am looking to schedule a task with bree every N seconds. N is chosen by the user. Let's say N=45s. This means the function should be called e.g. at 13:00:00,13:00:45, 13:01:30, 13:02:15, 13:03:00, ... Of course this becomes more complex for other Ns, such as N=167.

Ideally I could do it like this:

later = require('later')
const textSched = later.parse.text('every 45 sec');
function logTime() {
    console.log(new Date());
}
const timer = later.setInterval(logTime, textSched);

However, this does not actually run every 45 seconds, but it actually triggers every x:00 and x:45.

Is there any way to schedule something like this?

[niftylettuce]

{ interval: ms('45s') }, you could also use dayjs to build a date to start it at the next whole minute, e.g. { interval: ms('45s'), date: dayjs().add(1, 'minute').startOf('minute').toDate() }

[adrfantini]

Thank you for your prompt response!

Unfortunately this solution has the issue that it slowly drifts forwards. For N=5s and with a start at 00, it will log at 00, 05, 10, ... but after a while, since it drifts by a few ms every execution, it will possibly print at 01, 06, 11, ... etc.

Do you know of any solution to this?

[niftylettuce]

The better approach would be to use cron, also see #9

[niftylettuce]

Edit: Not #9, I meant look at local time support in https://github.com/breejs/bree/pull/76 and https://github.com/breejs/issues/74

[niftylettuce]

The drifting is most likely because of code execution time, perhaps there can be optimizations done in our source code further in bree itself.

[adrfantini]

The better approach would be to use cron, also see #9

Unfortunately from what I understand in CRON the same issue with values such as "every 45 seconds" occurs. Using step values (/45) causes CRON to run at 00:00 and 00:45, but not at 01:30.

The drifting is most likely because of code execution time, perhaps there can be optimizations done in our source code further in bree itself.

Indeed. This is the same behaviour of using chained setTimeouts (first example here: https://javascript.info/settimeout-setinterval#nested-settimeout). However for our usecase this is not acceptable. We need to run every N seconds, regardless of how long the task takes, not simply wait 10 seconds between executions. This is well explained in the link above. Chained setTimeouts with corrections for this are easy to implement, but it'd be nice to be able to use all the niceties of bree for this.

[niftylettuce]

Could you review our source code to check for places we could optimize to ensure accuracy?

[adrfantini]

It's not a matter of code efficiency. The issue is that bree uses setInterval or later.setInterval. Both tend to drift. If you run this code: setInterval(function() {console.log(new Date())}, 2000) you will get the following output:

2021-01-20T07:21:36.982Z
2021-01-20T07:21:38.985Z
2021-01-20T07:21:40.986Z
2021-01-20T07:21:42.988Z
2021-01-20T07:21:44.990Z
2021-01-20T07:21:46.992Z
2021-01-20T07:21:48.994Z
2021-01-20T07:21:50.995Z
2021-01-20T07:21:52.998Z
2021-01-20T07:21:54.999Z
2021-01-20T07:21:57.002Z
2021-01-20T07:21:59.003Z
2021-01-20T07:22:01.004Z
2021-01-20T07:22:03.006Z
2021-01-20T07:22:05.009Z
2021-01-20T07:22:07.010Z
2021-01-20T07:22:09.011Z

As you can see, it slowly drifts forward by a few ms every execution. After about 2-300 executions on my system, it will have drifted by a whole second (and now it executes on odd seconds instead of even seconds!).

This is well documented by several issues and blogposts. E.g.:

• https://github.com/nodejs/node/issues/21822
• https://gist.github.com/manast/1185904
• https://stackoverflow.com/questions/8173580/setinterval-timing-slowly-drifts-away-from-staying-accurate

One solution is to use recursive setTimeouts, which automatically compensate after each execution. Or use better timing functions.

Here are a few packages addressing this:

• https://www.npmjs.com/package/accurate-interval
• https://www.npmjs.com/package/tasktimer
• https://www.npmjs.com/package/nanotimer
• https://www.npmjs.com/package/self-adjusting-interval

[adrfantini]

So in short this is two issues in one:

1. not being able to specify intervals like 'every 45 seconds' and have it executed correctly every 45 seconds. This limitation is similar to an identical limitation in CRON.
2. If using ms('45s') or 45000, there is a drift due to the use of setInterval.

[niftylettuce]

Thank you so much for this @adrfantini. Excellent writeup. By chance are you interested in crafting a pull request to address this?

[adrfantini]

TBH I'm not really sure where to start (and I am a complete JS noob). I have no idea how to address 1.The way to address 2. would be I guess to not use setInterval and instead use some other more precise method