spirilis
commented
4 years ago @gerred mentioned on slack yesterday an option to have KUDO operators become consumers of CloudEvents, and use those to trigger plans - then externally to KUDO, or perhaps deployed by KUDO, have a CronJob or other system emit CloudEvents to trigger those events.
I think that would be a flexible way to implement this without messing with KUDO's parameter code and implementing crontab interpreters. There's probably a lot more use-cases we could implement with generic CloudEvent trigger capability. Possible ping-pong between plans (a plan triggers a CloudEvent which triggers another plan, etc).
Thoughts: CloudEvent triggers used by KUDO should have the option of a time-delay, either explicitly specified in the CloudEvent somehow or specified in the CloudEvent listener/consumer config. This is to avoid infinite loops where one plan CloudEvent triggers another which CloudEvent triggers another etc. in a tight resource-occupying loop.
What would you like to be added: Time-based triggering able to trigger based on specific days of the week, at specific times, similar to k8s CronJob
Why is this needed: KUDO is ideally suited for complex batch job execution - using Pipe-Tasks to create unique IDs and executing a sequence of pods referencing the unique ID which may be resource intensive (some may run in parallel, but some may be delegated to series steps to avoid blowing up cluster resource usage). However, we need a way to kick off the batch job on a deterministic timeline.
My current understanding of KUDO is that this requires manual intervention - either a crontab script on a Linux server executing "kubectl kudo" commands, or some kind of K8s CronJob pod with a copy of kubectl + kudo that kicks it off.
Details of the latter scenario - running a k8s cronjob in-situ - are murky to me at this point (what kind of clusterroles or roles would a pod need in its serviceaccount to execute kubectl kudo commands?) - but it also seems appropriate to me that batch jobs could have their crontab configuration included in the Operator, preferably as a Parameter itself that may be altered at runtime using kubectl kudo commands.
I envision this working like such: A crontab parameter (value of type string, using crontab-style syntax) is definable whose trigger only fires when the current date+timestamp satisfies the schedule stated in the crontab parameter string.
One alteration of KUDO's behavior for parameter-based triggers must apply here- This parameter DOES NOT TRIGGER when it is modified! It only triggers when the current date+timestamp satisfies its value's defined schedule. So the user can manually adjust this parameter at will with no consequence.