Scheduling

[Beormund]

Hi Peter,

I've been using your chron/scheduler successfully for a while now. Today I came across a slight issue. For a schedule: mins=range(0,60,15) and hrs=14; if you then run this at, for example, 15:20 the next run time produces 14:30 the next day instead of 14:00. But if you run it at 15:47 it correctly gives a next run time of 14:00. It seems to take into account the minutes past the hour for the current time's hour.

Thanks.

[peterhinch]

This flew under my radar because it was closed. I will investigate.

[peterhinch]

Thanks for the report - I can confirm the bug.

This is a use case that I hadn't considered: where a sequence is scheduled to take place at a future time. So far it's proving a knotty problem. I'll give it my best shot over the next few days, but it is possible that it may end up being documented rather than fixed.

If I can't find a reasonably "micro" fix, workrounds are possible. For example schedule a task for 14:00 which runs the payload, then schedules three repeats at 15 minute intervals.

[peterhinch]

There is a better workround which is to wait until just before the sequence is to start, then start it. The problem is that it looks ahead from the current time. If it sees a sequence it looks for the first entry after the current time. This is correct if the application has already waited on the cron instance, but not on initialisation.

I am leaning towards documenting this as fixing it without creating further corner cases is hard. By contrast the workround is easy.

I have written a simple script which allows sequences to be simulated to prove that they work as expected: users adapt the sim routine to match their proposed sequences. The following demos your case, with an initial delay, and shows that the sequence works correctly on the first and subsequent days.

# simulate.py Adapt this to simulate scheduled sequences

from time import localtime, mktime
from sched.cron import cron

days = ("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday")
tim = 0  # Global time in secs

def print_time(msg=""):
    yr, mo, md, h, m, s, wd = localtime(tim)[:7]
    print(f"{msg} {h:02d}:{m:02d}:{s:02d} on {days[wd]} {md:02d}/{mo:02d}/{yr:02d}")

def wait(cr):  # Simulate waiting on a cron instance
    global tim
    tim += 2  # Must always wait >=2s before calling cron again
    dt = cr(tim)
    hrs, m_s = divmod(dt + 2, 3600)  # For neat display add back the 2 secs
    mins, secs = divmod(m_s, 60)
    print(f"Wait {hrs}hrs {mins}mins {secs}s")
    tim += dt
    print_time("Time now:")

def set_time(y, month, mday, hrs, mins, secs):
    global tim
    tim = mktime((y, month, mday, hrs, mins, secs, 0, 0))
    print_time("Start at:")

# Adapt the following to emulate the proposed application
def sim(*args):
    set_time(*args)
    cg = cron(hrs = 13, mins = 59)  # Wait until 13:59 before starting sequence
    cn = cron(hrs = 14, mins = range(0, 60, 15))
    wait(cg)
    print()
    for _ in range(10):
        wait(cn)
        print("Run payload.\n")

sim(2023, 3, 29, 15, 20, 0)  # Start time: year, month, mday, hrs, mins, secs

Output (on a Pyboard) - payload runs at expected times:

Start at: 15:20:00 on Wednesday 29/03/2023
Wait 22hrs 39mins 0s
Time now: 13:59:00 on Thursday 30/03/2023

Wait 0hrs 1mins 0s
Time now: 14:00:00 on Thursday 30/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:15:00 on Thursday 30/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:30:00 on Thursday 30/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:45:00 on Thursday 30/03/2023
Run payload.

Wait 23hrs 15mins 0s
Time now: 14:00:00 on Friday 31/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:15:00 on Friday 31/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:30:00 on Friday 31/03/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:45:00 on Friday 31/03/2023
Run payload.

Wait 23hrs 15mins 0s
Time now: 14:00:00 on Saturday 01/04/2023
Run payload.

Wait 0hrs 15mins 0s
Time now: 14:15:00 on Saturday 01/04/2023
Run payload.

I will post an update in the week. I'll update the docs and post this script. I also plan to update crontest.py to make it more cross-platform.

[Beormund]

Thanks for looking into this. Does it only affect minute sequences? I don't seem to have any issue for hour lists but haven't checked hours with ranges/steps. I built a web UI around your scheduling utility so will give some thought about how to prepend the required crons for sequences:

[peterhinch]

It affects cases where a sequence is scheduled to start at a future time. I have pushed a draft release of SCHEDULE.md which explains the issue and suggests a workround. Comments and suggestions are welcome.

[Beormund]

It affects cases where a sequence is scheduled to start at a future time. I have pushed a draft release of SCHEDULE.md which explains the issue and suggests a workround. Comments and suggestions are welcome.

Mmm. If a user can create (and therefore initialise) a schedule at any time, and you can't rely on the next run time to be accurate for a sequence that might run in the future, how do you determine the delay_start schedule?

[Beormund]

I'll have a play with the code and see if reversing the processing order of time specifiers helps.

[peterhinch]

In the case of asynchronous code think I've found a solution. This makes no changes to cron.py which is a big benefit, as it is very hard to avoid creating new corner cases. I spent some time last week in the attempt before abandoning the attempt.

The idea is to use the techniques in simulate.py to calculate the time of triggers occurring after 0:00 on the current day. The aim is to calculate a time to wait before starting the sequence.

The first trigger occurring before the current time of day is t0, the first occurring after the current time of day is t1. Note that there may be no t0. In most cases you wait until (say) 5 minutes before the first trigger, then start the sequence. There are some corner cases such as fast sequences where this wait time needs to be shortened. I think these are straightforward and can be calculated from t0, t1 and the current time.

If I'm right this can be done by adapting schedule and be completely transparent to the user. I will start on this tomorrow.

[peterhinch]

I have pushed an update which aims to fix the issue in asynchronous code. The API is unchanged except that where an iterable is passed to secs, successive values must be at least 10s apart (formerly 2s).

Comments welcome :)

[Beormund]

I have pushed an update which aims to fix the issue in asynchronous code. The API is unchanged except that where an iterable is passed to secs, successive values must be at least 10s apart (formerly 2s).

Comments welcome :)

I did some testing. It's an elegant solution that works great - thank you! I extracted out the initial long_sleep (based on midnight last night) into a separate function - get_first_run(**kwargs) so I could call it from within my get_next_run() function.

def get_first_run(fcron, **kwargs):
    tim = mktime(localtime()[:3] + (0,0,0,0,0)) # Midnight last night
    now = round(time())
    while tim < now:
        tim += max(fcron(tim), 1)
    return tim - _PAUSE

async def schedule(func, *args, times=None, **kwargs):
    async def long_sleep(t):
        while t > 0:
            await asyncio.sleep(min(t, _MAXT))
            t -= _MAXT
    fcron = cron(**kwargs)
    first_tw = get_first_run(fcron, **kwargs) - round(time())
    await long_sleep(first_tw)    
    while times is None or times > 0:
        tw = fcron(round(time()))  # Time to wait (s)
        await long_sleep(tw)
        res = launch(func, args)
        if times is not None:
            times -= 1
        await asyncio.sleep_ms(1200)  # ensure we're into next second
    return res

# Gets a list of run times for given cron.
def get_next_run(**kwargs, reps):
    def next_run(c):
        t = get_first_run(c, **kwargs)
        def inner_run():
            nonlocal c, t
            _t = t+c(t)
            t = _t+1
            return _t
        return inner_run
    c = cron(**kwargs)
    f = next_run(c)
    return [f() for _ in range(reps)]

[peterhinch]

Thanks for testing. I have pushed a minor update which allows an Event instance to be passed as an arg in place of a callable. This is a trivial change which has no effect on operation with a callable.

I have formed a view that callbacks are rather clunky and am trying to add event-based options where possible. The docs explain this if you think you might apply it.

I think this effort is now complete, so I'll close this. Thanks for pointing out this nasty bug. Figuring out how to fix it took me down some long blind alleys, but the eventual fix is quite simple :) Now to tackle #101...