Adding more jobs in the cron run while its running (cron task queue with priority/weight)?

[klonos]

The idea of shortening the default cron run time to something like 10mins was mentioned in https://github.com/backdrop/backdrop-issues/issues/857#issuecomment-186698291. We do have cron running in the background now (#1214), but what happens when for example there are lots of cron jobs to be run that will take long time? Say these jobs are of minor significance (maintenance tasks like cache clearing or removing stale things etc.) compared to time-critical things like say having a node published in a specific point in time? How do we handle this? Could we give precedence to specific types of cron jobs/tasks?

[klonos]

So, I was thinking that perhaps we should introduce some sort of cron task "weight":

• We give cache clearing say a weight of 1.
• All other maintenance-related, routine tasks get a weight of 2.
• We reserve weight 0 for things like node publishing scheduling.
• Cron starts and gets a list of pending tasks.
• The task list is sorted by weight.
• Tasks with weight 0 are executed first.
• If there are many tasks with weight 0, then these are sorted say by date/time added.
• If there are no tasks with weight 0, move to tasks with weight 1.
• If there are no tasks with weight 1, move to tasks with weight 2.
• ...
• After each task is finished, resort/recheck the pending task queue by weight to check if any tasks with higher priority have been added in the meantime.

Doable? Thoughts?

[klonos]

...

• If there are many tasks with weight 0, then these are sorted say by date/time added.

Well, in the case of cron tasks for publishing nodes, perhaps instead of executing by order of time the task was added, perhaps we should be smarter and execute by time node is set to be published.

I think if all these features are implemented in cron, we should rename it to something "catchy" and market it as something special (it is!):

In Backdrop, the default cron shipped with Drupal was replaced by the next-gen Smart Cron that features background tasks so that webpages are served to visitors without interruption as well as tasks prioritization so that critical tasks are run on schedule.

[quicksketch]

Answering a couple of questions here:

• All cron runs have a check that prevent multiple jobs from running concurrently. A lock is acquired (or more accurately, we use state_set()) to indicate a cron job is running. If cron jobs overlap, the second one will simply not execute and it will wait for the next window.
• However... overlapping cron jobs are uncommon because PHP usually has a 60 second execution timeout (though this can be lengthened by Backdrop if needed). Cron jobs themselves are not possible to execute more than once a minute using the cron daemon, nor through the Backdrop UI. So once every minute is the theoretical shortest time without creating unusual setups.
• Different implementations of hook_cron() usually (but not always) set their own state_set() indicating the last time they were run, and only run once when that time window. For example you might run cron once every 5 minutes, but some jobs might only run once per day, regardless of how many times cron.php or the background cron is triggered.
• As for "cron weight", you can already determine the order in which your module executes cron relative to others by using hook_module_implements_alter() or setting your module weight in the system database table. Though that is for everything your module does within hook_cron().

Right now the way most modules implement cron they work on an assumption that they will not take longer than a few seconds to execute their particular task, and if other modules run their periodic tasks at the same time, they also won't take more than a few seconds. If a module does have a very long running task, then they use the Queue API to queue the tasks so that they can be run a the "next available cron cycle", because other tasks already might be queued before them that would need to be run first.

Overall, I think we already have most of the bases covered for what's requested here. Though a bad-behaving module might still use hook_cron() when a queue is more appropriate.

[klonos]

Thanx for taking the time to explain @quicksketch

I know, you'll hate me for the noise, but here's another though: if the default cron time is indeed shortened to 10mins or less, why not actually have it be running constantly (triggered at every page load/refresh) instead?

The reason why we had configurable time for cron AFAIK was in order to mitigate things like slow page loads. Well, we have no such issues in Backdrop anymore, since cron now runs in the background. So, why not run it all the time and see if there are pending tasks to be executed (useful for accuracy in #857)?

If we implement this, we can also lose the entire Cron config page (and the menu item from the admin bar) because:

• "Run cron" button will not be needed (any page refresh will trigger a cron run).
• "Run cron every" drop-down will not make sense (same reason - refresh the page instead).

We can also remove the entry from the site status report for UI simplification:

• No need to know when last cron was run (most likely the moment you visited that page - If site admins need to know when the last cron was run, we can always add "cron run successfully - x amount of tasks completed" entries in the watchdog log)
• No need to run cron manually (same reason as removing the button from the cron config page)
• No need to run cron from outside the site (less security checks/risks)

Again thoughts?

[ghost]

Again, I strongly disagree : all this annoying "automatic cron" stuff is here for sites which cannot (or do not want) use the host cron feature - this is badly named "from outside the site", but this is the only clean way to go. I'd like to remove all this stuff from core, and provide a contrib module ("Elysia Cron", do you remember...) for old low cost shared sites

[klonos]

... all this annoying "automatic cron" stuff is here for sites which cannot (or do not want) use the host cron feature... I'd like to remove all this stuff from core, and provide a contrib module ("Elysia Cron", do you remember...) for old low cost shared sites

Yes, but you are missing the point here (Backdrop CMS Philosophy):

Audience

Backdrop CMS is for the small to medium sized business, non-profits, educational institutions, and companies or organizations who are delivering comprehensive websites on a budget. ...

5. Meet Low System Requirements. Backdrop must be able to run on affordable hosting with very basic requirements....

(emphasis mine)

[ghost]

@klonos : no, I'm not missing the point : In my expression "old low cost shared sites", the key word is old : here in France, as of today, I can get a high quality "affordable hosting with very basic requirements" for less than 2€ / month (US$ 2¼), which includes CRON tasks, and free phone support if you have trouble setting it up…

[quicksketch]

Well, we have no such issues in Backdrop anymore, since cron now runs in the background. So, why not run it all the time and see if there are pending tasks to be executed (useful for accuracy in #857)?

I don't think there would be too much harm in this. As I noted above, your site is limited to a single cron execution running at any given time anyway. However, this may cause poorly written hook_cron() implementations to have exaggerated side-effects. Some cron jobs may delete all their caches, or rebuild an expensive operation, expecting that cron may be run no more often than every 5-10 minutes. Those are poor assumptions to be sure, but a bad hook_cron() implementation could start severely affecting the site by constantly flushing/rebuilding something that is supposed to be a long-term cache.

Overall, I think this may be a bit risky, but possibly not out of the question. For starters simply lowering the default interval to a shorter time (1-5 minutes) may be a good experiment.

Again, I strongly disagree : all this annoying "automatic cron" stuff is here for sites which cannot (or do not want) use the host cron feature - this is badly named "from outside the site", but this is the only clean way to go.

The ability to use a standard cron setup or to use a task-runner like Jenkins is completely valid. I think you're right that we definitely shouldn't remove the ability to disable the built-in cron. With our target audience however, I think that we should expect the built-in cron to be most-common, with explicitly set cron jobs or task runners as a common but not required configuration.

[quicksketch]

I'm not sure if any items are needed to be completed here. Even with things like file deletions, node scheduling, log cleanup, etc, we don't have any reports of cron failing at its jobs. For complicated task running, we should port Elysia/Ultimate Cron for Backdrop and leave it to contrib.

If there is an actionable item here, please reopen. Or let's use a new issue with a targeted objective.