Support Graceful shutdown

[graemerocher]

Feature description

Basically this is how it needs to work:

• When enable a filter is added that counts active requests with restart start/stop events
• When shutdown event is received start rejecting requests with SERVICE_UNAVAILABLE
• Then wait for requests to remaining requests to finish up until a defined timeout
• Then proceed with shutdown
• During this period the /health/liveness endpoint should continue to return UP
• During this period the /health/readiness endpoint should return SERVICE_UNAVAILABLE

Other considerations:

We may want to make the implementation generic and not coupled to HTTP For example you may want to wait for active Kafka message consumers to complete and then shutdown Or you may want to wait for active scheduled jobs to complete and then shutdown Implementation should there for be defined with a generic registry of counters:

interface ActiveResources {
    boolean hasActive(ResourceKind kind);
    void start(ResourceKind resource);
    void stop(ResourceKind resource);
} 

enum ResourceKind {HTTP_REQUEST, JOB, MESSAGE};

We could then later update the other implementations (Kafka, JMS etc.) to support it.

[yawkat]

it actually isn't possible to make this a generic filter because there's no api atm to know when a http request has been fully sent to the client (not just queued).