Scheduled maintenance tasks

[phgross]

Usecases GEVER:

• daily/weekly removal of trashed documents in resolved GEVER Dossiers, when Karrenzfrist is expired
• daily/weekly removal of the original file in resolved GEVER Dossiers, when Karrenzfrist is expired
• Consistency checks, such as searching for dossiers in non-leaf-repository-folders, and notifying when found (email?)

myteamraum:

• Scheduled sending of billing e-mails (once per month)

plone maintenance in general:

• Catalog: remove brains of missing objects.
• Catalog: scheduled/batched reindex of objects.
• Add-On cleanups, such as ftw.copymovepatches' fixes

Ideas, questions and thoughts

• Add-ons should provide their maintenance tasks.
• Maintenance tasks can be checks for errors and / or fixes for those errors.
• There should be a standardized way for notifying on (unresolvable) problems (email? slack?).
• For very long running tasks, e.g. reindex SearchableText for all objects, the system should be able to split that up into batches and run them over a period of multiple nights.
• Maybe we need to be able to schedule tasks on a specific date / time (business use cases), but we also need to have "general maintenance" tasks which should be run "sometime in a night, maybe once a week".
• It should be possible to schedule a job once, e.g. for the next night, such as complete catalog reindexes. This is invoked when we suspect something to be wrong and have a fix, such as reindexing, but it takes too long (conflicts) and should be done in the night.
• There should be a nice control panel, where we can see what did happen (executed jobs) and which jobs are scheduled.
• One-time-jobs could be triggered through the control panel.
• It would be awesome if periodically jobs could be registered in buildout, so that we can schedule it for a different time per deployment on the same server.

[Rotonen]

Initial thoughts:

• Zope's clock-server can be used to process maintenance tasks in guaranteedly small chunks at a high frequency
• I prefer cronjobs + bin/instance run for stuff which requires guarantees of when the jobs run
  • Cronjob management should be done via deployment configuration management tools
  • Ansible can be married into buildout - or the other way around
• There is no real need for an actual message queue based runner
  • This would complicate getting guaranteed feedback out of the tasks

There seems to be decent scaffolding to build upon: https://github.com/plone/ansible.plone_server https://github.com/plone/ansible.plone_server/blob/6b738ead23a5e499e35868ca2fc18c793ed4f241/tasks/main.yml#L351

[Rotonen]

https://pypi.python.org/pypi/slc.zopescript

Just spotted this: are we building our current ones on top of that or how are we rolling?

[Rotonen]

CastleCMS is using Celery:

https://github.com/castlecms/castle.cms/tree/cb8296e7a1c3daf0be46341a6e779868b6d34694/castle/cms/tasks

[jone]

@Rotonen We may also need a queueing / messaging solution for the SPV, depending in which path we walk. It would be great to have your opinion on that too: https://basecamp.com/2768704/projects/13482340/todos/313087237

[Rotonen]

@jone from my point of view these are the same thing - there is no sense in doing something which would not cover both.

[deiferni]

update from a quick discussion regarding consistency checks: also check expected service version (e.g. for bumblebee), add required service endpoint to do so if required

[Rotonen]

This seems to have been superseded by https://github.com/4teamwork/opengever.core/pull/5566 ?

[phgross]

With the Nightly Jobs feature (#5566) there is now a setup for nightly jobs. New jobs like the removal of the original files or trashed documents, is now possible by registering an additional JobAdapter.