ZeroAssumptions / aide-de-camp

Durable job scheduler for rust
Other
29 stars 2 forks source link

Graceful shutdown handling #15

Closed aschey closed 1 year ago

aschey commented 1 year ago

Hi, thanks for making this library. It's been really great to use so far. I've been tinkering with adding support for graceful shutdowns using Tokio's cancellation tokens in order to allow jobs to react to shutdown requests and perform any cleanup tasks before the server shuts down.

The basic flow is that the user can request the server to shut down by supplying an arbitrary future that will cancel the server on completion just like Hyper's with_graceful_shutdown method. The cancellation token gets propagated from the runner to the router's listen method and then to the process method so every component can react to the shutdown request. I also added some configuration options for consumers to optionally supply a cancellation timeout that specifies how long to wait for a graceful shutdown before the shutdown is forcefully completed.

Additionally, I added the cancellation token to the processor trait's handle method so that each job can easily react to the shutdown request in a standardized way. Now the method looks like this:

 async fn handle(
        &self,
        jid: Xid,
        payload: Self::Payload,
        cancellation_token: CancellationToken,
    ) -> Result<(), Self::Error>;

However, this is obviously a breaking change to the public API. I think it would be ergonomic for each job to have easy access to the cancellation request, but I can understand if you want to keep the handler method as simple as possible and leave it up to the consumer to propagate the cancellation request through some other means.

Let me know if you'd be interested in a PR for this and I can clean up my proof of concept and submit it for review.

andoriyu commented 1 year ago

@aschey was actually waiting on your PR before cutting a new release, so all breaking changes were in the same release. we would be interested in your PR and that's a feature we wanted to add ourselves.

aschey commented 1 year ago

Okay, great. I'll work on getting a branch together soon.