cylc / cylc-flow

Cylc: a workflow engine for cycling systems.
https://cylc.github.io
GNU General Public License v3.0
328 stars 92 forks source link

ICP = next(...) syntax improvement #3187

Open matthewrmshin opened 5 years ago

matthewrmshin commented 5 years ago

Issue discussed in this Discourse thread: https://cylc.discourse.group/t/cylc-question-on-wall-clock-synchronization/58

For example, in order to start a crontab job to run every 5 minutes starting from the earliest 5, 10, 15, etc minutes past the hour, the crontab syntax is just:

*/5 * * * * do-stuff

In Cylc, user has to do:

[scheduling]
    initial cycle point = next(T-00; T-05; T-10; T-15; T-20; T-25; T-30; T-35; T-40; T-45; T-50; T-55)
    [[dependencies]]
        [[[PT5M]]]
             graph = @wall_clock => do-stuff

It is fair to say that Cylc needs to do more than Crontab, but we should at least try to find a way to simplify the initial cycle point = next(...) syntax.

hjoliver commented 5 years ago

Just to note contemplation of cron-like real time scheduling in Cylc can get quite mind-bending when you consider things like: we could have parts of a cycling Cylc suite tied to a real time schedule while other parts (even with the same cycle points, potentially) are not tied; and note that ICP = now and ICP = next(...) choose ICP relative to real time "now" BUT tasks in the first cycle point will still trigger without regard for the clock unless entirely optional clock-triggers are attached to them.