Cron like scheduling primitives

[tillrohrmann]

It might be useful to be able to periodically invoke a function. The way such a cron function would be invoked is by an interval that is configured by the user.

[slinkydeveloper]

Related to https://github.com/restatedev/restate/issues/844 as well.

[slinkydeveloper]

We got some user feedback about this.

[slinkydeveloper]

Current proposal of the semantics for a first version:

• Schedules are expressed using the Cron format with second precision. This is an extension of the cron format used by crontab, used in a couple of projects such as Spring and Quartz.
• The cron is configured using an annotation on the handler, e.g. in Java/Kotlin @Scheduled(cron = "10 * * * * *"). The registration of crons is static and associated with the event of registering/updating a service.
• Only regular service handlers can be annotated with the @Scheduled annotation. Virtual objects and Workflows handlers cannot use it (for now).
• Through the CLI, a bunch of commands are exposed to observe the list of existing crons, the last X execution failures (if any).
• The default behaviour on missed ticks is to skip the invocation when the endpoint is down. In other words:
  • If the endpoint is down, or takes long to reply, and a tick passes, that tick is missed.
  • If the runtime is down, and at least one tick passes, the last tick should be executed. In practice, on recovery, Restate needs to figure out whether at least one tick elapsed, and needs to execute this.
• The handler MUST be empty request/empty response.
• On re-discovery, old crons are removed and new ones are registered. The first execution will be the next tick. In-flight invocations will continue to be processed by the old endpoint, as usual.

An extension of this proposal is to have an Admin API/CLI command or perhaps even Context API where crons can be dynamically created/removed, but perhaps we can do that once we get user feedback that this is effectively needed.

[ariskk]

This would be extremely useful.

The use case where we emulated scheduled services by implementing a recursive delayed object_send in a VirtualObject was pretty much as follows:

• Get the last processed state from context
• Check the state of an external system
  • If the two are different, trigger a method to process the new state
• Schedule another check by making a call to self with a 10 second delay

Slight difference here would be that the service would make a call to a VirtualObject holding the last processed state, but overall the flow could be easily implemented without the recursion hack.

Big fan of that feature! It would essentially enable Restate to sense (by polling) external systems that might not have the ability to push.

[geoff-nunan]

We are moving our BPMN execution engine over to restate. BPMN start events can be timer type start events and are configured using the ISO8601 repeating interval standard. https://en.wikipedia.org/wiki/ISO_8601#Repeating_intervals

Each time the timer triggers, it will call a service which will begin the execution of the BPMN process

[mupperton]

My use case would be a static cron, not dynamic on application events, to start a business process at the same time on the same weekday each week, e.g. 12:00 Monday

I will (future) need to be able to start the same business process at different times with different inputs, e.g. 12:00 Monday with ABC input and 13:00 Monday with DEF input. I see inputs are out of scope for current proposal, of course it's possible to define a dedicated handler for each to invoke the main handler with the desired input in the meantime. This in my case won't need to scale up to loads of different inputs

I would also like to be able to "stop" the business process from running some days on demand, and turn it back on again, dynamic to application events, but for this I'd probably want to just have a virtual object with some boolean state that acts as that decision gate, which the cron-invoked handler would first check before continuing. I would still expect the cron to receive a success response from the handler so it doesn't retry. Again, I don't believe anything in this paragraph needs anything specific built to support it, just context for you

Something worth considering: Are cron expressions going to be UTC, or will it be possible to specify a timezone location, e.g. "London" that auto switches between UTC+0 (GMT) and UTC+1 (BST)